IOM X applies Communication for Development (C4D) approaches to inform content and messaging to support positive behaviour change for the prevention of human trafficking and exploitation. IOM X follows a framework1 with inter-dependent steps to develop its evidence-based activities:
- Analysis
- Strategic Design
- Development and Testing
- Implementation
- Monitoring and Evaluation
This page gives an overview of the below steps and the research activities and methodologies used to inform and assess Open Doors:
1. Analysis
3. Development and Testing
5. Monitoring and Evaluation
Step 1: Analysis (Duration: 6 months)
Why domestic workers?
Domestic workers are at significant risk of exploitation because they work in the confines of a private home. They suffer from pervasive negative attitudes, and labour laws in many ASEAN countries do not cover domestic work. Despite the first global standards for domestic work created in 2011, domestic workers still commonly face abuses such as no weekly days off, excessive working hours, confiscation of identification, restriction of movement, salary deductions, non-payment of wages, and verbal and physical abuse
What did IOM X do?
IOM X developed a long-form video, Open Doors, to encourage employers of domestic workers to uphold fair labour standards in their homes. Open Doors is a three-part drama set in Malaysia, Thailand and Singapore.
Target audiences:
The primary audience is employers and those living in homes that employ live-in domestic workers. The secondary audience is domestic workers. Both audiences are based in ASEAN.
Objectives:
- After viewing Open Doors, viewers will have increased knowledge about the exploitation of live-in domestic workers and they will be motivated to adopt practices to reduce exploitation, such as providing one day off per week.
- For every activity that is assessed using KAP surveys (more details below), IOM X has the following cross-cutting objectives: at least 15% increase in knowledge of targeted audiences; at least 5% increase in positive attitudes; and at least 20% increase in positive intended practice.2
What research was conducted to inform Analysis (Step 1)?
Formative research was conducted to gain a deeper understanding of the relationship between domestic workers and employers, as well as understand employers’ knowledge, attitudes and intended-practices towards domestic worker rights.
| Research | Objectives | Respondents |
|---|---|---|
| Consultation with experts $ | Determine common forms of exploitation, which rights to prioritize and who should be targeted. | 10 experts on domestic worker issues |
| Focus group discussion (x1) $ | Understand motivations/ experiences of domestic workers. |
7 Myanmar domestic workers in Thailand |
| Online survey $ | Gain insights on attitudes towards domestic workers. | 57 youth (aged 18-25) primarily from the ASEAN region |
| In-depth interviews $ | Discover common working conditions, challenges, attitudes and experiences. |
6 employers of domestic workers and 16 migrant domestic workers in ASEAN |
The information gained from the formative research was used to inform Step 2 – the strategic design (the creation of a creative brief and draft communication materials).
[BOX yellow]
TIPS
Record interviews (make sure to get their consent!) so that you can pay attention to the
interviewees and listen again if you forgot what they said.
Record with mobile phones, dictation machines or software like Call Recorder.
Interview via Skype, WhatsApp, FaceTime and Viber to save time and cost. It’s also a great way of getting input from those who are not in the same city.
For online-surveys use free service providers such as Survey Monkey, which allow you to create and publish surveys, and to view your results graphically.
[END BOX]
STEP 3: Development and Testing (Duration: 1month)
To evaluate the effectiveness of the draft Open Doors scripts, they were pre-tested in focus group discussions with the target audience. A script reading by participants was facilitated (except in Thailand where the script was read by actors, filmed and screened to the focus groups).
| Activity | Objectives | Respondents |
|---|---|---|
| Focus Group Discussion (x 9) $$-$$$ |
|
91 employers and domestic workers in Bangkok, Jakarta and Kuala Lumpur |
The results from the testing were used to improve the programme, which allowed for Step 4 of the process, Implementation.
STEP 5: Monitoring and Evaluation (Duration: 4 months)
It is important to monitor and evaluate a program during and after its implementation, in order to assess its impact on the targeted audience, to learn what worked and what could be improved upon for future interventions.
Monitoring
Good practice involves routine data collection to measure the progress of an activity. Monitoring captures day-to-day activities to gain insights in to whether the target audience is engaging in activity (such as # of views) and what is happening to the target audience (such as text/content analysis of video comments). For a digital activity such as Open Doors, most of the monitoring involved online tracking.
[BOX yellow]
TIP
Behaviour change takes time and usually happens over a long period. However, as with any SMART objective, it is important to set a timeframe for data collection. This is necessary for budgeting reasons and to be able to deliver results to donors. For Open Doors, IOM X released a post-event report four weeks after the launch, which included the following monitoring data.
[END BOX]
| What is being monitored? |
What does this tell us? | 4 weeks following launch (25 June 2017) |
Latest statistics |
|---|---|---|---|
| Video views | Are people watching the video? | 1,526,446 | 76,922,347 (June 2017) |
| Visits to Happy Home landing page |
Are viewers taking steps towards learning more about the issue? | 6,204 | 358 average monthly visits (March 2017) |
Evaluation
To evaluate a programme’s impact, best practice is to conduct baseline research (before an intervention) and an impact assessment (after an intervention has been implemented). Useful evaluation tools are the Knowledge, Attitude and intended Practice (KAP) pre- and post- surveys by conducting surveys, which measure shifts in knowledge, attitude and intended- practice before and after an intervention. IOM X hired a research agency to conduct six KAP surveys with the primary target audience (employers of domestic workers) as well as members of the general public in three countries to inform the baseline and assess the impact of Open Doors.
KAP surveys are a standard methodology but it’s important that they are conducted in a way that is appropriate for the target audience of the intervention. As the targeted audience of this intervention have high access to the Internet, all surveys were completed online. Conducting surveys online also saves costs and are used when it is relevant for the target audience.
| Country | #of respondents and methodology | Demographics |
|---|---|---|
| Indonesia $$$$ |
Pre: 324 (April 2015) |
50% male, 50% female Ages: 15-50 62% employers of domestic workers and 38% general public (non-employers) |
| Thailand $$$$ |
Pre: 313 (April 2015) Post: 302 (June 2016) |
50% male, 50% female
|
| Malaysia $$$$ |
Pre: 208 (November 2016)
|
50% male, 50% female Ages: 15-50 100% employers of domestic workers |
KAP surveys can measure how successful an intervention was in accomplishing its objectives. The Open Doors KAP surveys measured levels of knowledge, attitudes and intended practices towards domestic worker rights based on a number of different questions/statements, which differed slightly based on country specific contexts. Some of these questions included were:
| KAP | Statement |
|---|---|
| Knowledge |
|
| Attitudes |
|
| Practices (intentions to do the following) |
|
The impact assessment showed that Open Doors succeeded in meeting most of its objectives:
1. Knowledge increased by an average 27% (target 15%)
2. Positive attitudes increased by an average of 21% (target 5%)
3. Intended practice increased by an average of 10% (target 20%)
A 20% increase in behavioural intent (intended practice) was not achieved because pre-survey levels were already high at 70%.
Learnings
Pre-testing draft scripts with employers and domestic workers proved to be extremely valuable as some of the focus groups did not recognize that the domestic worker (protagonist) was even a character in the script. Thus, IOM X collected suggestions from the focus groups on how to increase the visibility of the domestic worker character and these were shared with the creative agency to draft a new script. Pre-testing therefore ensured that Open Doors included messaging and storytelling relevant to the target audience.
[BOX yellow]
TIPS
- Some useful questions to keep in mind when monitoring programmes are:
- Is the programme reaching its intended audience?
- Is the target audience comprehending/relating to the content?
- What early signs of progress can we detect?
-
Websites such as Facebook and YouTube have their own analytics services for tracking engagement. For other social media, you can use free or paid accounts on Sprout Social to monitor engagement.
-
Large-scale KAP surveys like can be expensive. However, it is crucial to set aside budget for M&E activities to know if the intervention accomplished the desired impact.
- Even if the budget is tight, it is still possible to assess impact. To create a baseline, existing data such as the following can be used:
- Official statistics
- Existing survey results
- Research reports
- Journal and newspaper articles
- Your own survey created using Survey Monkey
Based on this information it is then possible to create a study to assess the impact of a programme.
[END BOX]
Conclusion
place over a two-year period. On average, IOM X assigns 12% of the total production budget to research and learning to ensure that the activity is providing IOM X with meaningful returns on impact and reach.
IOM X continues to disseminate Open Doors as it has proven to be an effective tool in terms of increasing knowledge about domestic workers’ rights and having a positive impact on respondents’ intended behaviour to adopt or to continue practicing positive behaviours towards domestic workers.
To learn more, watch Open Doors, read the impact assessment of the programmme.