Collecting Evidence

In this section, you will learn how to document local impacts and analyze whether the project has complied with certain standards, including international human rights standards, the environmental and social policies of development finance institutions, or industry standards. You will also learn how to organize and present the information to use in evidence-based advocacy.

Why collect evidence?

Recording and documenting the experience of the community you are supporting is crucial to successful advocacy. Collecting evidence and accurately reporting the harms that have occurred or are anticipated, and compliance with relevant standards, is important for several reasons:

  • It helps ensure that the voices and experiences of the community are at the center of your advocacy.
  • The process of explaining and discussing a community’s experience can help it become more organized and think through all of the impacts of the project, both good and bad.
  • This can help community members clearly articulate the problems they face and the solutions they seek.
  • It gives credibility to your advocacy and ensures that your claims are neither exaggerated nor understated.
  • It strengthens your advocacy by showing precisely how laws, policies, standards or codes of conduct have been breached.
  • It can form the basis of mediations or negotiations with responsible actors and an agreement for providing remedies and/or altering the project in a way that avoids harms and provides benefits to the community.
Collecting evidence involves gathering primary information directly from affected communities and other sources about the effects of the project on people and the environment. This information is typically presented in a report that includes a description of the project and the affected communities, the results of the research, and an assessment of whether human rights, laws or other applicable standards have been respected or violated. Reports that find shortcomings or violations usually end with a set of recommendations for the responsible actors to provide compensation and/or to change the design of the project.

Step 1:

Identify the main issues and impacts on the ground

To begin, conduct some preliminary scoping to identify the main impacts of the project. This initial scoping is necessary to design an appropriate assessment framework. This framework will help you design the data collection tools you will use when speaking to local communities and/or their representatives.

To start scoping out the issues, talk to a few people in the community about their experiences. Read media reports about the case. Try to find as much information as possible about the project, including, if available:

  • The business model of the project. For instance, if it’s an agriculture project, is it a large-scale plantation or a contract farming scheme?
  • Maps of the area that show the project’s location, boundaries and size. Does it overlap with land that is owned or used by local communities?
  • The type of project being developed and how it will affect the local environment and people. For instance, if it’s a mine, how will it affect the soil, local water sources and the environment?
  • Chemicals used or created. For instance, with a mining project, where will waste byproducts be disposed?

This type of information will help you start reflecting on the sorts of issues that are likely to arise in the implementation of the project. It helps to think about processes, gains, losses and local impacts.

Processes include things like when and how affected people first found out about the project; whether they were consulted and able to participate in the design; and other decisions about the project.

Gains include things such as access to infrastructure, water or economic benefits, such as new jobs.

Losses include things such as the land and natural resources taken or blocked by the company, in addition to houses, forests and crops destroyed.

Local impacts are the effects of the project on people’s lives, which might be linked to gains or losses, such as changes in:

  • the amount and quality of food they eat or are able to store;
    livelihoods, income, savings and debt levels;
  • physical and mental health;
  • personal safety and security;
  • children’s lifestyles, including school attendance and play; and
  • the ability to practice cultural or spiritual traditions.

Local impacts can be both positive and negative. For example, the incomes of some households may have dropped because of land loss and the destruction of crops or natural resources that they depend upon. At the same time, other households might have increased their incomes as a result of new jobs created by the project.
It is also important to find out about attempts by the community to express their opposition or concerns about the project, and the responses they received, including any compensation. There may have been intimidation, threats, violence or arrests directed at community members who expressed their opinions or tried to defend their land and resources. These should also be recorded.

It is important to understand if there has been any support for the project and why. Communities may in fact be divided over an investment project. Some may be against it, some may be for it, and others may not be opposed to the project but want to ensure that they benefit from it and are not harmed. Although the purpose of conducting this research is to support affected people in their advocacy (whether to stop the project, redesign it, or demand redress for harms suffered) it is important to have an idea of the different perspectives that may exist among those affected. This will help you anticipate how widely supported the community’s advocacy campaign will be, as well as possible responses and defenses put forward by actors along the investment chain.

At this stage, you do not need detailed data about the impacts of the project, just a general idea of what the main issues and impacts are or are likely to be.

Tip

Make sure the community is on board. When you speak with community representatives during the scoping process, use the opportunity to explain your intention to conduct further research and what this will involve. Make sure community members want your team to conduct this research and understand how the findings can be used to support their advocacy.

You should agree on a timetable for conducting interviews, including setting dates and selecting a time of the day that is convenient for them. It is also important to discuss any security measures that will be necessary for conducting the interviews. For example, if local authorities are likely to be hostile, you may decide to conduct the interviews in a private location or outside the community.

Step 2:

Develop an assessment framework

The next step is to develop an assessment framework to help you understand whether the project has complied with relevant standards. The framework will guide you in designing your questionnaires, structuring your report, and analyzing whether the project has breached the commitments and obligations of actors along the investment chain.

Many organizations choose to conduct human rights impact assessments because human rights standards are universal, bind governments, and are relevant to companies and financial institutions. Viewing the adverse impacts of projects as a violation of specific human rights can also be empowering for affected communities. Other frameworks used to analyze compliance include national laws, environmental and social policies of multilateral development banks, and company or industry standards and codes of conduct.

Tip

To develop an assessment framework, consider the main advocacy targets that you have identified in your investment chain analysis.

For example, if you have discovered that the International Finance Corporation has financed the project, it will be helpful to use its Performance Standards as your framework. If a company that has a strong code of conduct or internal policies is upstream or downstream along the investment chain, you might want to incorporate these policies into your framework. You may decide to use more than one set of standards that are applicable to your case. For example, you may have found links to the International Finance Corporation (IFC), but you may also think that using human rights and national laws in your assessment will be effective in influencing advocacy targets including the company, investors and governments.

Once you have chosen the set of laws, policies or standards to use for your assessment, you can start to develop your assessment framework. Identify the specific provisions or sections that are most relevant to the main issues and impacts on the community that you found during your scoping.

Tip
List the issues and impacts that you identified during the initial scoping. Underneath each of these, list the relevant provisions from the set of international and national laws, policies and standards that you’ve chosen to use. This is your assessment framework. It will guide you when you develop your questionnaires, organize your data and structure your report.
You will need to describe your assessment framework in the report, including an explanation of why you used the particular framework, who is bound by or has responsibilities under the framework, and the specific human rights, laws, and standards that are relevant to the case.

Step 3:

Develop a research methodology

Although you are collecting evidence to support community-led advocacy, it is important to be as unbiased as possible when conducting research. You will need to explain your methodology in your report in order to demonstrate that the results are credible. If you don’t, your advocacy targets may try to dismiss your findings as biased. It is therefore important to carefully design your methods to ensure that the data and information that you collect is accurate and objective. There is a range of tools that you can use to gather information about the impacts of the project. It is best to use a combination of tools to ensure your data is comprehensive, accurate and reflects the experiences of all groups within the community.

However, remember that if you use several tools, if your questionnaires are long, or if you interview a lot of people, you will end up with a large amount of information that you will need to organize and analyze. Where your research has been limited in some way — for example, due to limited resources or time, or because some community members declined to participate — you need to state this clearly in your report. This helps to build credibility by being open and honest about the possible limitations of your research and prevents other actors from discrediting your results.

Tip
Consider your team’s skills and experience to collate the data when you select and design your research tools. For example, is there someone on your team experienced in using computer software, such as Microsoft Excel, to organize and analyze large amounts of quantitative, or measurable, data? If so, you may decide to collect measurable information about impacts by a significant proportion of households. For example, you might collect data on the amount of land lost to the company and the amount of income lost per household. If your team does not have this capacity, you may decide to only collect qualitative, or descriptive, information and to keep your sample size, or the number of people you interview, relatively small. For example, you may decide to ask key persons and a small sample of households that have experienced serious impacts to describe the effects on their livelihoods and food consumption. The most interesting reports contain a combination of quantitative and qualitative information.

Data collection tools

Some of the main types of data collection tools you could use include: participatory mapping, key informant interviews, household or individual surveys, and focus group discussions.

  • Participatory community mapping can be a helpful exercise. You may wish to do this first, before conducting interviews. Key informants and others with knowledge about the village should be invited to participate. Ask participants to draw the main parts of the village, including its boundaries and main landmarks, on a flipchart. Ask them to draw different types of land uses on the appropriate parts of the map. Next, ask them to mark the boundaries of the project’s concession area to show where this overlaps with the village or community resources. This visual process can help both the research team and community members to understand the geography of the project and how it is affecting their village.
  • Key informant interviews are used to gather information about the community and impacts of the project. You might interview village elders, community leaders or organizers, local NGO staff working with the community, or even local government officials. These interviews are usually in-depth and generally aim to collect qualitative data — in other words, descriptive information that captures the project’s impacts and the community’s situation.
  • Household or individual surveys are used to collect in-depth information about impacts as experienced by people in the community. Like key informant interviews, these usually aim to collect qualitative data to describe the impact of the project on the community. However, depending on how you structure the questions, you might also be able to collect quantitative data. For example, if you have open-ended questions, respondents may give more detailed responses. If you have closed-ended questions, respondents will usually give a simple yes or no answer, or a number or ranking, such as severe, quite severe, positive, very positive. These closed-ended questions can be used to collect quantitative data.
Tip
You should pay special attention to vulnerable and disadvantaged groups within the community, such as the disabled, elderly or minorities that experience discrimination.
Make sure that you clearly explain the process you used to select the interviewees in your description of the data collection methodology in your report. The more people you interview from different households, community groups and demographic backgrounds, the greater the chance that your data will be regarded as credible and an accurate picture of what is happening on the ground. If you are analyzing whether human rights or other standards have been violated, it is important to find out about the worst issues and impacts experienced by people within the community. This means that you should intentionally identify and interview people who have suffered losses or negative impacts that may amount to violations.
  • Focus group discussions are another way to gather information about a project’s impacts. Some impacts are communal in nature and are best described in a group setting using open-ended questions. The questions aim to facilitate discussion among the group about particular issues and impacts. Interesting points can arise in a group discussion that are not always gleaned from individual household interviews.
  • Focus group discussions are useful for ensuring that impacts on women and children are recorded. In communities where men tend to dominate, separate women’s group discussions are an effective way for ensuring that women’s voices are heard. Focus group discussions are also useful for gathering information about distinct impacts of the project on minorities or marginalized groups, or on people with disabilities.
Tip
Although you should prepare some questions that get people talking about the right topics, you should also be prepared to ask unplanned follow-up questions to probe into unexpected or interesting issues that arise during the discussion. Do your best to politely encourage quieter participants in the group to share their thoughts. It is best to keep the number of participants to less than 12, so that everyone has an opportunity to express their views and experiences. It is particularly important to designate a separate note-taker in your research team to record what is said during the focus group discussions. This will allow the interviewer to concentrate on facilitating the discussion.
Designing questionnaires

Your team will need to prepare a list of questions for each data collection tool being applied. Your questions should be designed to collect in-depth information about the main issues and impacts set within your assessment framework. When developing your list of questions, start with each part of your assessment framework and think of the key questions the research team will need to ask in order to find out whether specific human rights, laws or standards have been breached.

Although the issues that you are investigating will be the same across all data collection tools, the ways you ask the questions will be different. For example:

  • When interviewing a key informant you might ask, “Has your community experienced a change in the amount and quality of food that it has access to because of the company’s project? If so, how has it changed, and why?”
  • When interviewing individual households or families, you might ask, “Before the project began, how many meals did your household normally eat each day?” You might also ask, “Now, how many meals does your household normally eat each day? Has the quality of food changed? If so, how?”
  • When conducting a focus group discussion, you might ask more open-ended questions, such as, “Describe any changes that your families and community have experienced in the food you eat since the company arrived.” As the discussion evolves you can help steer it in a particular direction, while also keeping the discussion to the point.
Tip
Questions should not be leading. You should not assume particular positive or negative impacts, but instead ask neutral questions. For example, start by asking whether and how things have changed, rather than what the negative impacts have been.
See the Case Study on the Human Rights Impact Assessment of Rubber Plantations in Ratanakiri, Cambodia for a sample questionnaire.

Conducting and recording interviews

In addition to taking written notes of respondents’ answers, it is important to record all your interviews and discussions so you can check and clarify information that you have missed in your notes. It is also important to have recordings so that if anyone challenges your data, you have proof of what was said during the interviews. Audio recorders are inexpensive and easy to use and are a worthwhile investment for conducting research. Many smartphones also have a recording function. Recordings will also allow you to find good quotes from interview respondents to use in your report.

Make sure you ask the people participating for their consent to record conversations. Let them know that you will not publish their identities unless they consent. If they do not want to be publicly identified, you will record their names for your own verification purposes only, and you will not make them known to anyone outside of the research team. If someone does not agree to being recorded, you must respect their wishes.

During your visits to the community and to the project site, take photos if possible. Visual evidence is very effective at informing the audience and influencing your advocacy targets. If possible, you can also record interviews by video camera that show impacts of the project. Make sure you ask people for their consent to photograph or film them and explain to them how the photo or film will be used. People may be concerned that if their identities are revealed to governmental officials or the company, they will suffer reprisals. Err on the side of caution if this is a concern and respect the wishes of those who do not want themselves, their families, or property filmed or photographed.

Step 4:

Organize and present your data

Once you have completed your data collection, you will need to organize your data. This process will depend on the type and amount of information collected by the team.

If your team includes experienced researchers with access to software, this process could involve entering the data into the program, which can then derive various types of statistics. If you collected only qualitative data from a smaller sample size, this process may simply involve reading through your notes and listening to interviews, marking and separating information about each issue and impact, and then describing or summarizing the information under the headings in your assessment framework.

Use your assessment framework to organize your information. This will allow you to easily structure your report in a logical way. 

Inserting direct quotes from affected people, case studies that describe an individual or a family’s particular experience, and photos are great ways to make the report more interesting and informative and to ensure that voices from the community are heard.

Tip

Remember that you may need to omit or change names of individuals or families in order to protect them from possible reprisals. However, you should reference the place and date of interview, unless this information could put people at risk. Make sure you obtain the informed consent of any individuals who are identified or identifiable in the report.

If you are using statistics, inserting graphs, charts and tables can make the report more visually interesting and easier to understand. Maps of the area and of the village can also be very helpful to the reader.

There are several different ways you can structure your report. See the Case Study on the Human Rights Impact Assessment of Rubber Plantations in Ratanakiri, Cambodia for an example structure.

Step 5:

Analyze compliance

For each section of your report, review your data and analyze whether it shows violations of the relevant human rights, laws and standards in your assessment framework.

Rather than just making a finding that the project has not complied with a particular standard, analyze how the right, law or standard was violated, pointing to specific aspects or provisions. Try to attribute responsibility to particular actors. For example:

  • If you find a violation of a particular human right, the company’s action may have directly caused the violation, while a responsible government authority may not have taken measures to prevent the company from taking those actions, failing in its obligation to protect against human rights violations by third parties.
  • If you find that a particular section of a national law or regulation was breached, note who had responsibility for complying with that provision.
  • If you find that a particular part of an International Finance Corporation’s Performance Standard was not met, responsibility can be attributed to both the company for failing to comply and to the International Finance Corporation for failing to properly supervise its investment.
Remember to also make findings on notable positive impacts or ways in which rights and laws were respected or fulfilled and standards were met. This ensures the report is balanced and not seen as biased.
The conclusions of your analysis will be used in letters, media and complaints. They will serve as the backbone of your evidence-based advocacy campaign.

Step 6:

Verify your data

Verifying — or fact-checking — your data is essential for compiling evidence and for ensuring that the community is engaged and aware of the content of the report.

Verify your data by presenting the information and findings in your draft report to affected people in a form they can easily understand. Check with them that each piece of information is accurate. Following the verification session, make any necessary corrections and clarifications to your draft. You can also use the sessions as an opportunity to collect any missing information or to obtain additional quotes and case studies.

Consider whether it is a good idea to send the draft report to the company and/or government authorities at this stage to obtain their feedback. Although this is an important part of verifying information and ensuring your report is unbiased, you may need to weigh this against security concerns and the likelihood that the company and government will contribute constructively to the fact-finding process.

You should consider whether sending the government and company the draft report will be beneficial or detrimental to your advocacy efforts. If you send them the draft, their communication staff will have plenty of time to influence media coverage of the final report, and this could mean that the launch of the report loses some of its impact. However, if you do not send them the draft for feedback and include their response in the report, they may use this against you by trying to discredit the findings as one-sided.

Step 7:

Formulate recommendations based on your findings

The main purpose of collecting evidence is to support the community in seeking accountability and remedies for harms suffered or anticipated. Setting out clear recommendations to each responsible actor is an important way to make the research useful.

The report should contain recommendations for each advocacy target identified in your investment chain and correspond with their responsibilities under international human rights law, national law and/or standards to which they have committed. If possible, recommendations should also correspond with the degree of responsibility each actor has for the adverse impacts, and the leverage they hold to ensure harms are remedied and the project is redesigned to bring benefits to the community.

Tip
Recommendations should be realistic for the responsible actor to adopt and should reflect the desired outcomes of the community. Recommendations can be discussed during the validation sessions with the community to ensure that they reflect their aspirations.

Using your report in advocacy

When your report is complete, you should have concrete evidence on which to base your advocacy. There are many ways you can use your report to bolster your advocacy. You can organize a press conference with community representatives to launch the report. You can send a copy of the report to all relevant government agencies, the company and other actors along the investment chain. If there were violations of national laws, you can use the findings as the basis of a complaint to the courts or to other grievance and accountability mechanisms, such as the International Finance Corporation’s Compliance Advisor Ombudsman, the relevant Organization for Economic Cooperation and Development National Contact Point or the Special Procedures of the United Nations Human Rights Council. You can also use the findings to strengthen your position in negotiations with the company and other responsible actors, and as a basis for formulating remedial solutions.

For a full discussion of how you can use the evidence you have collected to bolster the community’s advocacy, please see the Advocacy Strategies section of this guide and sub-sections on Engaging Advocacy Targets, Using Courts, Using Non-Judicial Grievance Mechanisms, and Using Human Rights Mechanisms.
See the Case Study on a Human Rights Impact Assessment of Rubber Plantations in Ratanakiri, Cambodia for a detailed examination of how Inclusive Development International and our partner Equitable Cambodia collected evidence for one case.