Search Results for: handwashing

Global Handwashing Day

October 15 is Global Handwashing Day!

Washing your hands is one of the most effective ways to prevent the spread of germs.

Studies have shown that handwashing can prevent 1 in 3 diarrhea-related sicknesses and 1 in 5 respiratory infections.

Learn how to properly wash your hands with soap and water, and show others how to practice good hand hygiene.

Participate in Global Handwashing Day

For Global Handwashing Day on Wednesday, October 15, let’s commit to promoting proper handwashing and hand hygiene! Here are some ways you can participate:

Use these sample social media posts along with the graphics on this page and the hashtags #GlobalHandwashingDay and #foodsafety.
Share the videos (see above) that encourage proper handwashing and food safety steps.
Share the handwashing coloring placemat and the handwashing coloring page on social media with the hashtags #GlobalHandwashingDay and #foodsafety.
Watch the replay of the webinar Clean Hands, Healthy Home! which highlighted observational research on consumer handwashing, and how handwashing awareness and the recent shifts in handwashing behavior. CEUs are available!

Handwashing Mythbusters

Common food safety myths originate from the misapplication of science, family tradition, or misinformation on social media.

The Partnership created these social media graphics for consumers and educators to help debunk common home food safety myths.

Brush up on safe food handling steps with these food safety Mythbusters!

English | Español

Clean Hands, Healthy Home!

Just in time for Global Handwashing Day, health, nutrition and food safety educators are invited to download and share these resources on social media and in your outreach.

The infographic (right) is available as a JPG file and printable PDF file.

These colorful social graphics teach families how to properly wash hands while preparing food and handling raw meat, flour and eggs.

Use these sample social media posts along with the graphics on this page. Let’s help educate families on proper hand hygiene!

Teach Kids Proper Handwashing

Food Delivery

Know your role! As the way we receive food changes, the core food safety practices remain essential to protecting our health and the health of those we love.

Washing your hands is one of the most effective ways to prevent the spread of germs. Wash your hands with clean, running water and soap for at least 20 seconds before and after handling food. If soap and water aren’t available, use an alcohol-based hand sanitizer that contains at least 60% alcohol.

Learn more about the three simple steps you need to follow to keep your delivered food safe!

Free Downloads

Food safety starts at home! Check out these free resources to help keep families healthy while preparing food at home.

The Core Four Practices

Coloring Pages for Kids

Safe Poultry Handling

The Story of Your Dinner

Kids Activities

These kid-friendly activities will help teach young ones how to properly wash their hands, as well as other food safety steps to help keep them healthy!

Scrub Club

Food Safety Activity Sheets

Coloring Placemats

Games & Recipes

Handwashing Tips

Washing your hands is one of the most effective ways to prevent the spread of germs. Studies have shown that handwashing can prevent 1 in 3 diarrhea-related sicknesses and 1 in 5 respiratory infections.

Follow the CDC’s five steps for effective handwashing.

Wet your hands with clean, running water (warm or cold), turn off the tap, and apply soap.
Lather your hands by rubbing them together with the soap. Lather the backs of your hands, between your fingers, and under your nails.
Scrub your hands for at least 20 seconds. Need a timer? Hum the “Happy Birthday” song from beginning to end twice.
Rinse your hands well under clean, running water.
Dry your hands using a clean towel or air dry them.

If soap and water aren’t available, you can use an alcohol-based hand sanitizer that contains at least 60% alcohol.

For more information visit www.cdc.gov/handwashing.

Additional Resources

Global Handwashing Day was founded by the Global Handwashing Partnership, and is an opportunity to design, test, and replicate creative ways to encourage people to wash their hands with soap at critical times. Global Handwashing Day is celebrated every year on October 15.

Handwashing for Ordinary People in a Time of Novel Coronavirus

March 11, 2020

Have you been hearing in the news, at school and at work that there are important things you need to do to stay healthy?

BTW, the coronavirus handwashing advice you’ve been hearing about has always been recommended for your healthy daily life!

That’s right, these handwashing basics should be a part of your daily life and your family and friends too! Here’s the Handwashing How!

How should I wash my hands?

Check it out — there are 5 easy steps.

Wet your hands with warm running water and apply soap.
Rub your hands together to make a lather and scrub them well. Be sure to scrub the backs of your hands, between your fingers and under your nails.
Continue rubbing hands for at least 20 seconds, about the time it takes to sing the “Happy Birthday” song twice.
Rinse your hands well under running water.
Dry your hands using a clean cloth or paper towel.

This video from the U.S. Centers for Disease Control and Prevention (CDC) gives you all the basics. Share this video with everyone you know.

So now you know how.

When should I wash my hands to protect myself and others from the risk of harmful germs?

Well, there are many times throughout the day, including:

Before eating food
Before, during and after preparing food
Before and after treating a cut or wound
Before and after caring for someone who is sick
After changing diapers or cleaning up a child who has used the toilet
After handling uncooked eggs or raw meat, poultry or seafood (or their juices)
After blowing your nose, coughing or sneezing
After touching an animal or animal waste
After touching garbage
After using the toilet or assisting another with doing so

Also, consider that your smartphone or tablet could be a source of contamination. If you use your device in the kitchen while preparing food, you should wash your hands after holding or touching the device.

What should I use to wash my hands?

Research shows that “plain” soap is just as good as any other soap. Using soap and water is the best option.

If running water is not available for handwashing, then use a hand sanitizer as a back-up. Your hand sanitizer product should be at least 60% alcohol content in order to be effective.

Finally, it is important to take time to help young children wash hands properly.

I know, I know. You’re thinking: I can’t keep up with all the times during the day my child should wash his or her hands.

Still, take time to model proper handwashing. The research about the benefits of regular handwashing says it all. Modeling proper handwashing sets kids up for a healthy life!

Download a free poster or this Happy Hands song placemat to remind your kids of the 5 steps to proper handwashing!

Happy Handwashing!

Shelley Feist is the Executive Director with the Partnership for Food Safety Education. She can be reached at (202) 220-0651 or sfeist@fightbac.org. Connect: LinkedIn | Twitter

Could I Have Food Poisoning?

October 15, 2023

How do you know if you have food poisoning?

Foodborne illness, commonly called “food poisoning,” affects about 1 in 6 people in the United States each year, according to data from the Centers for Disease Control and Prevention (CDC).

Symptoms can range from mild to very serious, with approximately 128,000 hospitalizations and 3,000 deaths per year caused by foodborne illness in the U.S.

Five Most Common Foodborne Pathogens

So, how do you know if you might have food poisoning? Below, you’ll find a table with symptoms for the five most common foodborne pathogens that cause illnesses. However, please note that only a healthcare provider can confirm whether or not you have a foodborne illness.

Pathogen	When Symptoms Appear	Common Symptoms
Norovirus	Generally 12-24 hours after exposure	Nausea, vomiting, diarrhea, abdominal pain. Headache and low-grade fever may also accompany this infection. Lasts for 24 to 60 hours.
Salmonella	Generally 8-12 hours after eating	Diarrhea, fever, abdominal cramps, vomiting.
Clostridium perfringens	Generally 8-12 hours after eating.	Abdominal pain, diarrhea, and occasionally nausea and vomiting. Symptoms up to a day and are usually mild. Can be more serious in older or debilitated people.
Campylobacter	Generally 2-5 days after eating.	Diarrhea, abdominal cramping, fever, and sometimes bloody stools. Lasts 7-10 days.
Staphylococcus aureus	Generally 30 minutes to 8 hours after eating.	Diarrhea, vomiting, nausea, abdominal pain, cramps, and prostration. Lasts 24-48 hours. Rarely fatal.

You can find a more detailed list of foodborne pathogens and their symptoms on our website.

What should you do if you have food poisoning?

Did you have any of the symptoms above? What should you do now? Don’t panic! Here’s a list of steps you can take:

Seek immediate help

Get help from a healthcare provider immediately if you are pregnant or if you have any severe symptoms, including:

Bloody diarrhea or diarrhea that lasts more than three days
Fever higher than 102°F (38.9°C)
Vomiting so often you can’t keep liquids down
Signs of dehydration, including not urinating much, dry mouth and throat, or feeling dizzy when standing up

Prevent dehydration

Drink plenty of fluids, especially if you are experiencing diarrhea or vomiting to prevent dehydration, which is a lack of water in the body.

Avoid spreading germs

Wash hands and surfaces often. Wash your hands with soap and warm water for 20 seconds before and after handling food, as well as after using the bathroom, changing diapers, and handling pets.

Preserve the evidence

If a portion of the suspect food is available, wrap it securely, mark “DANGER,” and freeze it. Save all packaging materials, such as cans or cartons. Write down the food type, the date, other identifying marks on the package, the time consumed, and when the onset of symptoms occurred. Save any identical unopened products.

Record foods eaten in the past seven days.

If you or your healthcare provider suspects foodborne illness, make a list of everything you ate in the seven days before symptoms began. Symptoms can appear anywhere between four hours to one week after ingesting a contaminated food item. This information may assist in determining the cause of your illness, and can help the health department verify if your illness is part of an outbreak.

Call the local health department

If the suspect food was served at a large gathering, from a restaurant or other food service facility, or if it is a commercial product, contact your local health department to inform them.

Request laboratory testing

If you or your doctor suspect a foodborne illness, it is important to run tests to determine which pathogen caused your illness to ensure you get the correct care and treatment. Identifying the pathogen can help treat you now and help you understand what issues may arise in the future.

What can you do to help prevent food poisoning?

Here are five easy steps you can take now to help prevent foodborne illness:

1. Wash your hands.

Studies have shown that handwashing can prevent 1 in 3 diarrhea-related sicknesses and 1 in 5 respiratory infections. Wash your hands with warm water and soap for at least 20 seconds before and after handling food and after using the bathroom, changing diapers, and handling pets.

2. Keep up to date on food recalls.

Recall notices can be found in the news, at your local grocery store, or online at www.recalls.gov. If you determine that a food recall product is in your home:

Don’t eat the food.
Don’t open the food.
Check the recall notice to find out what to do with the food

3. Practice the core four

When preparing meals at home, follow these core four food safety principles: clean, separate, cook and chill. Learn more about the core four food safety practices and how these practices can help prevent foodborne illness.

4. Cook with safe recipes

Safe recipes include food safety steps, like washing your hands, that lower the risk of getting foodborne illness. Studies have shown that when people follow recipes with basic food safety steps included, they are more likely to practice those steps. You can find delicious safe recipes to download, or you can turn your own recipe into a safe recipe using the Safe Recipe Style Guide!

5. Spread food safety awareness

Talk to your friends and family about how to prevent foodborne illness. You can tell your family about the core four food safety practices, test your friends’ knowledge of common food safety myths, or teach kids how to Fight BAC-teria. Follow us on YouTube, Instagram, or Facebook to stay up to date.

Curricula and Programs – K-3

Smart Kids Fight BAC! ®

Smart Kids Fight BAC!® is an award-winning curriculum for Grade K-3 consisting of an animated video; teacher guides; activity booklets that reinforce food safety through math, language arts, art and science; and He’s BAC! – a storybook for children and parents. The curricula was created by Dr. Judy A. Harrison, Professor of Foods and Nutrition and Extension Foods Specialist at the University of Georgia. Dr. Harrison currently serves on the board of The Partnership for Food Safety Education.

Smart Kids Fight BAC! Level 1 Activity Book – For classroom and after-school use. Puzzles, games, activity coloring pages for the younger kids. Suitable for grades K-3.

Smart Kids Fight BAC! Level 1 Teaching Guide – Guide for educators on how to use Smart Kids Fight BAC! Level 1 activity book. Answer keys and pre and post tests. Suitable for grades K-3.

Smart Kids Fight BAC! Level 2 Activity Book – For classroom and after-school use. Puzzles, games, activity coloring pages at elementary school level. Suitable for grades K-3.

Smart Kids Fight BAC! Level 2 Teaching Guide – Guide for educators on how to use the Smart Kids fight BAC Level 2 activity book. Answer keys and pre and post tests. Suitable for grades K-3.

He’s BAC! – He’s germy, he’s green, he’s very rude. Check out this great illustrated storybook “He’s BAC” written by Judy A. Harrison, PhD, The University of Georgia, with illustrations by Jeff Fastener.

The original Fight BAC!® education program for young children to teach them about the four basic concepts of food safety.

About the Fight BAC! Program for Grades K-3 – A hands-on guide to the interactive kid-friendly Fight BAC! education program.
Planning to use the Fight BAC! Program for Grades K-3 – A guide to the planning process for PFSE’s Fight BAC! education program from the right time to implement to the proper preparation.
Fight BAC! Presentation – A guide to executing PFSE’s Fight BAC! education program: Storytelling techniques, frequently asked questions by kids and sample responses. Suitable for grades K-3.
The Story of BAC – A script to introducing students to the BAC character and teaching them the basic fundamentals of safe food handling. Suitable for grades K-3.

Coloring Sheets

BAC with Cookie Dough

BAC with Turkey

Certificate of Participation

Handwashing

Hot and Cold

Keep Kitchen Clean

No BAC Logo

Perishables

Picnics

Recipes

Food Safety Mistakes

Fruits and Vegetables

Refrigerator

Safe Handling Label

Wash and Dry

Other Coloring Sheets

The Story of Your Dinner Placemats

Other Activity Sheets

Web-Based Games

Data Collection

How to collect data

In addition to planning and selecting your evaluation design, you also need to figure out how to collect evaluation data. There are two main types of data: quantitative and qualitative.

Quantitative data is quantifiable and numerical and is particularly useful when trying to establish causality between an independent variable (program or activity) and dependent variable (food safety knowledge, attitudes, or behavior etc.) or if you want to obtain some sort of score or rating on a topic, such as a knowledge score. It is best to use quantitative methods when the subject topic is well researched and when you have a valid and reliable data collection tool. This method is usually considered to be more objective and less biased than qualitative methods. In addition, it can be easier to demonstrate the validity and reliability of quantitative data than with qualitative data.

Qualitative data is generally non-numerical and more exploratory in nature. It is used to identify important themes related to a particular topic and to gather detailed insight into more complex issues [11]. Qualitative data collection methods can provide valuable information about personal thoughts, experiences, feelings, and interpretations that can often be overlooked when using quantitative methods. It is a particularly good method to use when there is little is known about research topic.

Below are different data collection methods you could use to for your evaluation. Consider the benefits and limitations of each option in relation to your resources, the purpose of your evaluation, and your target audience.

Collection Method	Description	Benefits	Limitations
Survey/ Questionnaire	• Self-reported. • Usually a series of questions that can be provided online or on paper to collect numerical data. • It is important to use a survey that has been pre-tested and proven to be reliable and valid.	• Good for collecting quantitative data that can be statically analyzed. • Good for assessing food safety knowledge. • Inexpensive and requires less time, staff training, and support. • Can be a good method to use to establish causality between an independent variable (program or activity) and dependent variable (food safety knowledge, attitudes, behavior etc.).	• May overlook deeper personal meanings attitudes or perceptions related to food handling and why people think or behave the way they do. • Risk of self-reported biases.
Focus groups	• Self-reported/descriptive. • Usually consists of a group of 8 to 12 individuals that come together to answer questions and have a discussion on pre-determined topics as a collective. A facilitator is usually present to facilitate dialogue and guide the discussion. Descriptive data is collected and later analyzed. • It can be helpful to keep some questions open-ended in order to gather information that is relevant and important but that may not have been considered when focus group questions were developed. • Focus group discussions are often recorded and later transcribed. It can also be beneficial for the facilitator to take notes on relevant nonverbal expressions to supplement recording transcriptions. • Most common method for collecting qualitative food safety information [14].	• Good for collecting qualitative data. • Can provide valuable information about thoughts, attitudes, perceptions, experiences, values, personal interpretations, and meanings that can often be overlooked when using quantitative methods. • Useful to explore topics on which little is known. • Interactions between group members may provide valuable insight on the topic, which can be overlooked when focusing only on individuals.	• Can be a less expensive and less time consuming way to collect qualitative data. • Need a skillful facilitator to encourage a productive discussion. • Can sometimes face scheduling difficulties with finding a time suitable for all participants to meet [17]. • Analysis can be time consuming. • May be difficult to find participants that are willing to openly share their personal thoughts and feelings in a group setting. • One or a few individuals may dominate the discussion so it is important for the facilitator to encourage equal participation. • Risk of self-reported biases.
One-on-one interview	• Self-reported. • Interviewer usually meets one-on-one with the interviewee either in person or via phone to ask pre-determined questions. Usually takes longer than when providing a written questionnaire. • Can be good to develop an interview training and script for all interviewers for consistency. • May be helpful to keep some questions open ended in order to gather information that may be relevant and important but that may not have been considered when focus group questions were developed.	• Structured interviews can be good for collecting quantitative data with additional insight into why participants respond the way they do. • Good for collecting qualitative data. • Useful method to explore topics on which little is known. • Can provide valuable information about thoughts, attitudes, perceptions, experiences, values, personal interpretations, and meanings that can often be overlooked when using quantitative methods [17].	• May be difficult to find participants that are willing to openly share when one on one. • Can require a skillful interviewer to encourage a productive discussion or response to questions, particularly with more sensitive topics. • Risk of self-reported biases.
Household audit	• Observed behavior. • An audit tool is generally used to visually examine and score households based on factors related to safe food handling practices. For example, an audit can examine resources needed for proper cleaning, cleanliness of the kitchen, or storage of foods [14]. • It is important to use an audit tool that has been pre-tested and proven to be reliable and valid.	• Observing food safety behaviors might provide more accurate and objective information than when relying on self-reported information [14]. • Can be a good complement to other forms of data collected, such as self-reported data.	• Participants may not be willing or feel comfortable allowing auditors to come into their homes. • Participants may prep their home before the audit, making the household environment less realistic (social desirability). • An audit score may not provide the entire picture for why participants scored the way they did. • Can be subject to rater bias if some auditors score more lenient or harsher than others.
Observations in model or consumer home or kitchen	• Observed behavior. • Participants are observed practicing a behavior or carrying out a specified task in a model or consumer home or kitchen.	• Observing food handling behaviors, might provide more accurate and objective information than when relying on self-reported information [14]. • Can be a good complement to other forms of data collected, such as self-reported data.	• Can be difficult to implement, time consuming, and expensive. • May be difficult to find participants willing to be observed when demonstrating food safety behaviors.
Collect microbial data in homes or kitchens	• Microbial samples are collected in the participants’ homes or kitchens and then taken to a lab and analyzed.	• Provides quantifiable data and information that cannot be collected via other quantitative and qualitative methods. • Can be a more objective method of collecting food safety information. • Can be a good complement to other forms of data collected, such as self-reported data.	• Does not directly provide information on food safety behavior or KASA. • Can provide insight on the presence and persistence of pathogens in domestic kitchens that can be valuable in developing recommendations for safe food handling practices at home [14]. • Participants may not be willing or feel comfortable allowing data collectors to come into their homes.

Mixed methods

Consider a mixed methods approach and using a combination of data collection methods to evaluate your program and gather insight into food safety knowledge, attitudes, and behavior. For example, collecting data via a questionnaire to assess food safety behavior and digging deeper into the topic via focus groups can provide a more well-rounded picture of outcome changes instead of solely relying on data from the questionnaire. Using qualitative methods to supplement quantitative methods can provide more background information on the topic and a greater understanding about why individuals responded the way they did quantitatively. Using mixed methods can also help you identify inconsistencies or inaccuracies when having to rely on self-reported data.

Self-reported data

When collecting self-reported data on food safety behaviors it is important to minimize potential threats to validity, such as recall and social desirability bias, to ensure that the data collected is reliable, consistent, and true. If possible, consider also using an observational method to collect the same behavior data in order to compare and analyze potential discrepancies between self-reported and observed behaviors. Below are examples of how studies have used mixed methods to gather food safety data:

To identify sanitation and food handling of ‘Chicken and Salad’ in Puerto Rican households, food and kitchen surface microbial samples were collected at different stages of food preparation. In addition, household observations were collected to observe storing, thawing, handling, and cooking practices. Observations and microbiological results were then compared to understand the impact of different food handling practices and risk of microbial contamination [13].
To learn about how consumers prepare and cook ground beef for hamburgers, video footage of 199 volunteers in Northern California were analyzed for compliance with recommended practices. Following the filming of each session, questionnaires about food safety attitudes and knowledge were provided to each volunteer. When describing findings from the video observations, researchers provided further insight into why participants engaged in specific practices in the videos by providing personal statements [22].
To explore home food safety knowledge, practices, and risk perception among Mexican-Americans, ten focus groups with 78 participants were conducted in New York and Texas. Focus group findings were then used to inform a probability-based survey that was administered to 468 Mexican-Americans who cook for their families. Findings from the focus groups and online surveys consistently identified several food safety concerns such as low use of thermometers, knowledge gaps about cross-contamination, and unsafe thawing practices [21].

Data collection tools

When thinking about what data collection tool to use it is important to do some research to find out if any tools have already been developed, validated, and used to address a topic similar to yours, with a similar population. Instead of starting from scratch consider using existing tools or adapting them to suit your needs. Below are examples of existing food safety tools that have been tested for validity:

Audit tool for domestic kitchens [3,6]
Food safety psychosocial questionnaire for young adults [7]
Stages of change questions to assess consumer readiness to use a food thermometer when cooking small cuts of meat [23]
Food safety knowledge and attitude scales for consumer food safety education [20]
Checklist for observing food safety behavior for sample young adults [5]
Consumer food behavior questionnaire [19]

Instrument/survey development

If you are developing your own instrument and questions for the evaluation, there are many things you should keep in mind such as including demographic questions, the length of your survey, health literacy and cultural sensitivity, and more. Below is a list of helpful tips to guide you throughout the instrument development process:

Include demographic questions in your survey. Responses can later be analyzed to find out how different characteristics influence food safety knowledge, attitudes, and behaviors etc. Gathering demographic information can also help you figure out who your program works best for. Alternately, you might find out that your program doesn’t work well for a group of individuals and you may need to adjust the program slightly for a certain group. You could also find out that you need to provide different versions of your activities or materials for different groups. For example, you may find that some messages are resonating well with females but not as well with males and you need to review and adjust materials given out to male participants. Collecting demographic information can also be helpful when analyzing data because you can adjust for certain characteristics to reduce threats to validity.
- Demographic information you may want to ask for include: age, gender, ethnicity, education level, number of individuals or children in household, and income. Some questions might be more sensitive in nature, so it may be helpful to restate that responses are confidential and anonymous (if that is the case) before asking sensitive questions such as income. It may also be beneficial to leave sensitive demographic questions towards the end of the assessment to allow participants to warm up and feel more comfortable before having to respond to more personal questions.
When using quantitative data collection methods make sure you use measures that are sensitive to change and can provide you with sufficient and useful information on the topic. For example, using a 5-point Likert scale can usually provide more valuable information on a topic than when using dichotomous variables such as yes or no options [9].
Provide the option “I don’t know” so participants are not forced to pick a response that might not be true for them.

Sample questions using a Likert scale and an “I don’t know” response option: The next few questions are about how confident you feel about carrying out different food handling practices to prevent foodborne illness. Please circle how much you agree with the next set of statements.

Consider the length of the survey and the time it takes a person to complete it. Keep surveys administered by an interviewer to 15-20 minutes and self-administrated surveys to 5-10 minutes [9].
When possible, consider using previously validated surveys or survey items and scales such as those provided in the “Data collection tools” section. Using a collection of questions or scales that have been previously validated can help save you time and reduce threats to internal validity.
When assessing food safety knowledge, use learning objectives to develop questions. For example, if the learning objective of a workshop or lesson curriculum is “participants will understand severity and susceptibility of foodborne illness” then start there to identify corresponding survey questions such as, “what are some of the consequences of foodborne illness?” or “what populations are most vulnerable to foodborne illness?”
Remember health literacy and cultural sensitivity. Make sure questions are clear, direct, and easy to understand, and that you take into account reading levels of participants. When thinking about cultural sensitivity, consider data collection methods that are culturally appropriate, applicable to the target audience, and sensitive to cultural norms. For example, some populations might be more receptive to female interviewers or feel more comfortable being interviewed by individuals from their own community or that speak their native language. Take time to learn and understand what interview strategies will work best for your target audience.
Pilot test your survey instrument. Adjust and refine your tool based on feedback and reactions. Consider using cognitive interviews and the think aloud method when pilot testing [9]. Cognitive interviewing is a technique that allows individuals to verbalize their feelings and thought processes [2]. Find out whether or not, and how, participants comprehend the questions, are able to retrieve information for their answer, judge whether or not their information is an accurate or relevant answer, and respond to the question [10,12,16,18,26]. Incorporate open ended probes such as “what thoughts are going through your mind right now?” or “what could we do to improve this question?” to gather feedback and reactions to the questions [9].
Test for reliability to ensure your tool will provide consistent responses and results when used repeatedly. For example, assess reliability by administering the same test to the same individuals over a period of time to ensure consistency in the results.
Test for validity to ensure that your tool actual measures what it is supposed to measure. There are four types of internal validity usually measured when creating an instrument [15].

Types of validity –

Face validity: the degree to which an instrument appears to measure the concepts or constructs you wish to measure. This can be the weakest form of validity because it is subjective and not evidence based. You could assess face validity with a group of stakeholders or representatives of the target audience by asking if the group finds that the questions are relevant and address the constructs or topics you are interested in.
Content validity: is the degree to which measures in the instrument contain a reasonable number of attributes from the concept you wish to gather information on. This can be measured by having a panel of experienced judges identify all the attributes of a concept and assess how representative the measure is of the concept, such as by using as 1-9 scale with one being extremely inappropriate and 9 being extremely appropriate.
Criterion validity: is the degree to which a measure can accurately predict the dependent variables or the outcomes you want to gather information about. It can be found by comparing a measure with other measures to find how much they correlate with each other. For example, you could measure how much an individual’s food safety skill score correlates with his or her actual ability to implement safe food handling practices. A measure has criterion validity if a high correlation is found with at least one of the criterion measures.
Construct validity: refers to how much an instrument is able to assess the theoretical construct or concept, such as self-efficacy, it is meant to measure. Construct validity is present when measures of an instrument are consistent with the theoretical hypotheses of the concept you want to gather information on [8].
Provide training for inexperienced interviewees and opportunities for them to practice and role play with representatives from the target audience. Prepare a data collection manual that describes procedures and provides information such as background on the program, recruitment methods, and data collection schedules, procedures, materials, and submission requirements [9]. Provide each interviewer with his or her own manual and go over its contents during training. More details on what to include in a data collection manual can be found on page 45 of the USDA’s Addressing the Challenges of Conducting Effective Supplemental Nutrition Assistance Program Education (SNAP-Ed) Evaluations: A Step-by-Step Guide: http://www.fns.usda.gov/sites/default/files/SNAPEDWaveII_Guide.pdf
Include an interview script to introduce and conclude the survey. Also include instructions and explanations that will help guide interviewee’s and participants through the questions.
When introducing the survey, provide an explanation on why it is important and how the participants’ feedback can help their community.

Sample: “Thank you for participating in this interview. Your feedback will help us learn more about food safety education and how we can improve our program to best serve your community and reduce foodborne illness. The purpose of this interview is to find out what you know about food safety, how you feel about food handling practices, and how you store and prepare foods at home. Remember, there are no right or wrong answers, so please feel free to say anything that comes to mind.”

Make sure questions are relevant and address your evaluation objectives.
Think about whether you are assessing inputs, outputs, and outcomes when developing evaluation questions. Below are examples of input, output, and outcome questions to evaluate a social media campaign on Twitter [4]:
- Input: How many pilot tested Twitter posts have been developed?
- Output: How many messages were posted throughout the campaign (October-December)? How many Tweets were retweeted? How many Tweets were clicked as a favorite? How many new followers were gained?
- Outcome: How many teenagers in the county learned about the new Germ Wars campaign?

To evaluate a health education initiative in Georgia elementary schools, pre- and post-test questionnaires were developed to evaluate the program’s effectiveness in increasing knowledge about proper handwashing. Keeping the target audience in mind, evaluation questions were developed to use a similar format to those often used in elementary schools. Extension food safety educators and child development specialists examined evaluation questions for content validity and readability and to ensure they were appropriate for the age- and grade-level of the sample. The University of Georgia Institutional Review Board approved all evaluation methods and instruments. Questionnaires were distributed in test packets students were accustomed to using when taking standardized tests. Evaluation data, collected from 5,462 youth, indicated that the program materials were effective in increasing knowledge about handwashing.

Harrison, J. (2012). Teaching children to wash their hands – wash your paws, Georgia! Handwashing education initiative. Food Protection Trends. 32(3), 116-123.

Ways to administer a questionnaire

Ways to administer a questionnaire	Benefits	Limitations
Mail	• Participants might find it more convenient to have the ability to take the survey at their own pace and time and in their own home. • Reduces the need for participants to travel and can avoid transportation challenges. • Does not require interviewers, which can save time and less staff support.	• Need a list of mailing addresses. • Printing and mailing may be costly. • Unable to track or confirm whether surveys were actually received. • Participants might ignore the mailed questionnaire and not respond. • Cannot track how long it takes participants to complete the survey. • Can miss visual cues and reactions that might be valuable and informative. • Participants might go online or ask someone for assistance with answering questions and there is no way to track this. • Participants might not complete and mail back surveys in a timely fashion.
Email or Web-based Communication	• Can be a convenient and a good option for a tech savvy audience. • Participants might find it more convenient to have the ability to take the survey at their own pace and time and in their own home. • Reduces the need for participants to travel and can avoid transportation challenges. • Low cost to set up and maintain. • Easy to administer and requires minimal on the ground staff support. • Ability to track percentage of emails that are viewed and monitor status of the survey using programs such as Qualtrics (https://www.qualtrics.com/) or Survey Monkey (https://www.surveymonkey.com/). • Can reduce time needed to input survey responses into a spreadsheet as survey programs will provide that service.	• Need to have a list of emails or be able to contact participants to send the survey or a link to the survey. • Need to ensure participants are comfortable taking online surveys (may not be best for older populations). • Need to ensure participants have reliable access to Internet and a computer or smart phone. • Participants might go online or ask someone for assistance with answering questions and there is no way to track this. • Can miss visual cues and reactions that might be valuable and informative.
In person group-administered	• Participants might feel more comfortable taking the survey in a group setting than one-on-one. • Can take up less time and require fewer interviewers by implementing all at once. • Can be easy to implement following a program activity such as workshop, training, or event, when all participants are in the same location at the same time. • Participants can share concerns or ask questions in real time. • Allows for a more personal experience and interviewer can document visual cues and reactions.	• Need to be able to find a time and location where participants are all together – may be difficult if the program does not already provide opportunities for this to take place. • Can require more resources (staff, time, and transportation).
In person one-on-one	• The target audience may prefer more personal face-to-face interactions. • Allows for a more personal experience and interviewer can document visual cues and reactions. • Participants can share concerns or ask questions in real time.	• If going door-to-door – participants may not feel comfortable allowing a stranger into their home or open their door for someone they don’t know. • Can require more resources (staff, time, and transportation). • Can face scheduling difficulties.
Phone	• Participants might find it more convenient to be interviewed in their own home or any location of their choice. • Can be easier to schedule. • Reduces the need for participants to travel and can avoid transportation challenges. • Participants can share concerns or ask questions in real time.	• Need a list of phone numbers. • Must ensure participants have access to phones. • Participants may not feel comfortable answering an unfamiliar number. • Can miss visual cues and reactions that might be valuable and informative.

Recruitment and Retention

Another factor that contributes to the success of an evaluation is participation. Below are some tips to help you recruit and retain participants:

Incentives, Incentives, Incentives! Provide incentives to participants who complete the evaluation. Do this each time you administer an assessment (at the pre-test and the post-test). Emphasize the incentive opportunity as you recruit and advertise. Involve members from the target audience when selecting the incentive (gift card, discount, coupon, or freebies) to make sure it is something that people actually want and will motivate them to participate.
Use key informants or individuals from the target audience to help with the recruitment process.
Be flexible when scheduling and plan around the participants’ availability.
If possible provide services such as transportation, refreshments, or day care.
Be open and honest when explaining the purpose of the interview. Keep it short and let people know the process won’t take more than X amount of their time.
Always be respectful, friendly, and maintain a good reputation in the community or with your target audience. Listen and be attentive to questions or concerns. Creating and maintaining a good reputation can help ensure that people are willing to work with you for future opportunities as well.
Say thank you! Always thank participants for their time and valuable feedback.
Don’t lose touch with participants if you need to do a follow-up test. Ask for the best way to reach them and get in touch in advance to inform them of the next survey time.
Follow through with any promises or commitments you make to participants. For example, if you advertise a specific incentive as a thank you gift, make sure the same incentive is provided to participants. Or, if you tell participants they will be contacted within a week with an answer to their question or to follow up on the evaluation process, make sure it is done so within the promised time frame.
Keep participants in the loop. Let them know if or when you will be sharing evaluation findings. Consider providing an open presentation of the final data for any interested participants and allow them to invite friends or family. This can help the target audience feel more actively involved in the process and can demonstrate how important and valuable their feedback is for the program.

Ethics

Throughout the evaluation, and even the needs assessment, it is important that you think about protecting the rights of the participants. This is particularly significant when interacting with vulnerable populations which include children under 18, prisoners, pregnant women, or anyone who is at risk of being coerced. Make sure that you are sensitive to the culture, needs, and rights of the target audience throughout the program implementation and evaluation, instead of focusing solely on evaluation or research goals.

Consider using a community-based participatory research approach to your program and evaluation. This approach involves forming collaborative partnerships between researchers and community members to ensure equitable involvement throughout the process. This can be a great way to empower your target audience to be active participants throughout the needs assessment, implementation of the program, and the evaluation. A community-based participatory research approach can also help you be aware of any ethical concerns throughout the program and evaluation and learn about how to best address any potential challenges. Working closely with your target audience as an equal partner can also build valuable trusting relationships that are beneficial to both parties and can foster co-learning [25].

Note: It is important to ensure confidentially of all the information that is collected from participants, particularly if confidentially is promised to participants during recruitment. When planning your evaluation think about what steps must be taken or systems need to be in place to ensure confidentially throughout the data collection and analysis process.

Internal Review Board

The U.S Department of Health and Human Resources defines research as “a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge” [24]. If your evaluation research fits this definition and if you plan on sharing or publishing evaluation findings as generalizable knowledge then you will need to apply for Internal Review Board (IRB) approval [17,24]. You will also need approval if you are receiving any kind of Federal funding. The purpose of an IRB review is to ensure the protection of the rights of human subjects and participants in research. To obtain IRB approval you will need to submit an application to a local or private IRB committee and demonstrate that you will be following Federal guidelines, such as those related to research ethics and informed consent.

The length of time it takes to obtain approval generally depends on factors such as the sensitivity of the topic, the target audience, and the level of risk involved with the research. There are 3 types of internal review you may be eligible for [17]:

Exempt – No risk or less than minimal risk to participants
Expedited – Minimal risk to participants
Full review – More than minimal risk to participants

Informed consent

Whether or not you need to obtain informed consent depends on how you plan to use the evaluation data and the requirements of your organization and/or the program or evaluation funders [17]. If you are required to obtain informed consent, here’s what you need to include in the informed consent form [17]:

Consider the following best practices when obtaining informed consent to make sure participants fully understand their rights and the information they are consenting to [1]:

Recognize the importance of time – do not make the informed consent process too long.
Train staff on the importance of informed consent.
View and treat participants as part of the decision making process.
Consider your audience: tailor the informed consent process to address cultural differences, health literacy levels, language needs, and demographic factors.
Use plain and simple language and provide information at an 8th grade reading level or below.
Think about using alternative methods to convey information such as video, visual handouts, or PowerPoint.
Assess and confirm comprehension by using the “teach back” or “teach to goal” method. This involves participants saying back to you the information you shared with them until they demonstrated that they fully understand the information.

Health Insurance Portability and Accountability Act

You must also ensure that you do not violate any Health Insurance Portability and Accountability Act (HIPPA) laws when conducting any research or your evaluation [17]. HIPPA protects an individual’s right to keep their health care information private. Depending on the information collected the evaluation, you may need participants to sign a form providing permission for you to share their medical information [17]. HIPPA regulations generally apply to healthcare organizations that provide medical services, which might not always be applicable for consumer food safety education programs [17]. However, it is important to keep HIPPA in mind if you plan to ask questions related to the health status of participants or the kinds of health services they have received.

In Summary,

when thinking about data collection and how to apply what you learned in this chapter to your program you may want to ask:

How will evaluation data be collected (e.g. questionnaire, focus group, or observations)? What are the benefits and limitations of this data collection method?
Can mixed methods be used in the evaluation?
Do any evaluation instruments already exist that could be used or adapted for the evaluation?
If a new evaluation instrument will be developed –
- Will demographic questions be included?
- How will the instrument be pilot tested?
- How will health literacy and cultural sensitivity be taken into account?
- How long with the survey be?
- What will I ensure validity and reliability of the tool?
If administering a questionnaire – how will it be distributed? What are the benefits and limitations of this method?
What will I do to recruit and retain participants?
When do I need to submit the IRB application by? What type of internal review is my evaluation eligible for?
What will I do to ensure confidentially of information collected?
What will I do to ensure program evaluation methods are ethical?

References

Aldoory, L., Ryan, K.B., & Rouhani, A. (2014). Best practices and new models of health literacy for informed consent: review of the impact of informed consent regulations on health literate communications. Institute of Medicine. Retrieved from: http://www.nationalacademies.org/hmd/~/media/Files/Activity%20Files/PublicHealth/HealthLiteracy/Commissioned-Papers/Informed_Consent_HealthLit.pdf
Beatty, P. C., & Willis, G. B. (2007). Research synthesis: The practice of cognitive interviewing. Public Opinion Quarterly, 71(2), 287-311.
Borrusso P, Quinlan JJ. (2013) Development and Piloting of a Food Safety Audit Tool for the Domestic Environment. Foods, 2(4),572-584.
Brodalski, D., Brink, H, Curtis, J., Dia, S., Schindelar, J., Shannon, C., & Wolfson, C. (2011). The health communicator’s social media toolkit. Electronic Media Branch, Division of News and Electronic Media, Office of the Associate Director of Communication at the Centers for Disease Control and Prevention (CDC). Retrieved from: http://www.cdc.gov/healthcommunication/toolstemplates/socialmediatoolkit_bm.pdf
Byrd-Bredbenner, C., Maurer, J. Wheatley, V., Cottone, E., & Clancy, M. (2007). Observed food safety behaviors of young adults. British Food Journal, 109(7),519-530.
Byrd-Bredbenner, C., Schaffner, D. W, & Abbot, J. M. (2010). How food safe is your home kitchen? A self-directed home kitchen audit. Journal of Nutrition Education and Behavior. 42,286-289.
Byrd-Bredbenner, C., Wheatley, V., Schaffer, D., Bruhn, C., Blalock, L., & Maurer, J. (2007). Development of Food Safety Psychosocial Questionnaires for Young Adults. Journal of Food Science Education; 6(2),30-37.
Carmines, E. G., & Zeller, R. A. (1979). Reliability and Validity Assessment. Thousand Oaks, CA: Sage Publications.
Cates, S., Blitstein, J., Hersey, J., Kosa, K., Flicker, L., Morgan, K., & Bell, L. (2014). Addressing the challenges of conducting effective supplemental nutrition assistance program education (SNAP-Ed) evaluations: a step-by-step guide. Prepared by Altarum Institute and RTI International for the USDA, Food and Nutrition Service. Retrieved from: http://www.fns.usda.gov/sites/default/files/SNAPEDWaveII_Guide.pdf
Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12, 229-238.
Creswell, J. W. (2007). Chapter 3: Designing a Qualitative Study. Qualitative Inquiry and Research Design: Choosing among Five Approaches, 35-41.
Daugherty, S. D., Harris-Kojetin, L., Squire, C., & Jael, E. (2001). Maximizing the quality of cognitive interviewing data: An exploration of three approaches and their informational contributions. Proceedings of the Annual Meeting of the American Statistical Association.
Dharod, J. M., Perez-Escamilla, R., Paciello, S., Venkitanarayanan, K., Bermudez-Millan, A., & Damio, G. (2007). Critical control points for home prepared ‘Chicken and Salad’ in Puerto Rican households. Food Protection Trends, 27(7), 544-522
Food and Drug Administration (FDA). White Paper on Consumer Research and Food Safety Education. (DRAFT).
Grembowski, D. (2001).The practice of health program evaluation. London, U.K.: Sage Publications.
Haeger, H., Lambert, A. D., Kinzie, J., & Gieser, J. (2012). Using cognitive interviews to improve survey instruments. Indiana University Center for Postsecondary Research – Paper presented at the annual forum of the Association for Institutional Research. Retrieved from: http://cpr.indiana.edu/uploads/AIR2012%20Cognitive%20Interviews.pdf
Issel, L. Michele. (2014) Health program planning and evaluation: a practical and systematic approach for community health Sudbury, Mass.: Jones and Bartlett Publishers.
Jobe, J. B. (2003). Cognitive psychology and self-reports: Models and methods. Quality of Life Research, 12, 219-227
Kendall, P. A., Elsbernd, A., Sinclair, K., Schroeder, M., Chen, G., Bergmann, V., & Medeiros, L. C. (2004). Observation versus self-report: Validation of a consumer food behavior questionnaire. Journal of Food Protection, 67(11), 2578-2586.
Medeiros, L.C, Hillers, V. N, Chen, G., Bergmann, V., Kendall, P. & Schroeder, M. (2004) Design and development of food safety knowledge and attitude scales for consumer food safety education. J Am Diet Assoc. 104(11), 1671-1677.
Parra, P. A., Kim, H., Shapiro, M. A., & Gravani, R. (2014). Home food safety knowledge, risk perception, and practices among Mexican-Americans. Food Control, 37(1), 115-125.
Phang, H. S., & Bruhn, C. M. (2011). Burger preparation: what consumers say and do in the home. Journal of Food Protection, 74(10), 1708-1716.
Takeuchi, M. T., Edlefsen, M., McCurdy, S. M., & Hillers, V. N. (2006). Development and validation of stages-of-change questions to assess consumers’ readiness to use a food thermometer when cooking small cuts of meat. Journal of the American Dietetic Association, 106(2), 262–266.
U.S Department of Health & Human Services. (2009). Code of federal regulations, title 45, public welfare, part 46 protection of human subjects. Retrieved from http://www.hhs.gov/ohrp/regulations-and-policy/regulations/45-cfr-46/index.html
Wallerstein, N. B., and Duran, B. (2006) Using community-based participatory research to address health disparities. Health Promotion and Practice. 7(3), 312-323.
Willis, G. B. (1999). Cognitive interviewing: A “How To” guide. Research Triangle Institute. 1999 Meeting of the American Statistical Association. Research Triangle Park, NC: Research Triangle Institute. Retrieved from: http://appliedresearch.cancer.gov/archive/cognitive/interview.pdf