This weekly blogging reflection is due as a threaded blog post below by Thursday, February 27 @ noon. No exceptions for late work, except with Dr. W's consent.
Read our HANDS ON SOCIAL MARKETING book, assignment above.
In a SINGLE blog post below for ALL chapters in the section, provide for EACH chapter:
1. A single sentence, IYOW, that captures the THESIS (main argument) for each chapter.
2. TWO specific pieces of supporting documentation - ideas, concepts, steps - to bolster your thesis for each chapter. (Use 2 - 3 sentences for each.)
3. A SINGLE specific question you have after reading and blogging on ALL chapters of the section.
Game on,
Dr. W
Section VII
ReplyDelete22. This chapter emphasizes the importance and basic steps of evaluation of your social marketing campaign.
22a. Evaluation benefits you more than your funders. It creates accountability, and will help improve the program while it’s in implementation.
22b. There are several types of evaluation. Questions need to be asked: What should we do? How are we doing? Did we do it? You need formative evaluation, process evaluation, and summative evaluation.
23. The chosen research method requires an evaluation design.
23a. Using data from existing records is less time-consuming and beneficial for your evaluation. There are several elements of the evaluation design, like setting goals and objectives, assessing the data to be collected, methodology, data collection instruments, data processing and analysis, and evaluation report.
23b. For smaller programs, same-group pretest-posttest is effective. It’s also helpful to use control groups. Evaluation indicators will tell you if you have achieved the right social marketing objectives. Some community based indicators can be related to environmental change, policy and regulation, information accessibility, and behavioral outcomes.
Question: What would be the best evaluation design for a larger scale social marketing program?
24. This chapter describes the possible evaluation methods you can use for your program.
24a. Surveys are the most popular method used for social marketing programs. Observation studies are also useful and require a systematic approach.
24b. Qualitative methods allow you to understand your campaign effects from different perspectives. This includes in-depth interviews, focus groups, and anecdotal feedback mechanisms.
25. The overall purpose of the evaluation is to take the collected feedback and make necessary changes to improve your program.
25a. Real-time feedback is improving the program as it’s in action and quickly implementing your feedback. It’s important to focus on strategic, rather than exploratory formative research.
25b. The collected feedback will also give you lessons for the future. Some lessons include running the program more smoothly, seeing the elements that didn’t work, noticing the objectives that need more effort, locating effective channels, correctly budgeting time and money, and what kind of challenges to expect.
Section VII
ReplyDeleteChapter 22: Evaluation Basics
Thesis: The focus of this chapter is the importance of evaluating social marketing campaigns.
A: When questioning what we should do, it’s important to have a formative evaluation. Formative research is a method that trial the campaign prior to application.
B: There are many challenges when it comes to evaluation. Some of these challenges include setting unrealistic expectations, having limited resources, reliance of a sing method and using the wrong model.
Chapter 23: Evaluation Design
Thesis: The focus of this chapter is the use of evaluation design.
A: When starting to gather data, a useful tactic is looking into seeing if other research has been gathered on this topic that can be used. Compare your data to this information for accuracy.
B: When evaluating there are some key indicators that will tell if you have achieved your objectives. Some of these include environmental changes, policy and regulation, information accessibility and behavioral outcomes
Chapter 24: Evaluation Methods
Thesis: The focus of this chapter is the use of different evaluation methods.
A: A widely used method for evaluation is doing surveys. Some examples of popular surveys include mailed questionnaires, telephone interviews, in person interviews and computerized surveys.
B: Another route to take when trying to evaluate is the use of qualitative methods. Some useful questions to consider when doing this includes what elements of the campaign have you seen /heard, what do you think about the campaign and what do you like or dislike about the campaign?
Chapter 25: Using Feedback to Improve Your Program
Thesis: The focus of this chapter is how to use the feedback you’ve received to better the campaign.
A: Some questions to think about when considering improvements for the future include how to make the social marketing process run more smoothly, which elements of the campaign worked and which didn’t, finally which objectives require additional effort or a new approach.
B: When implementing some of the ideas learned from the evaluation it’s important to consider the following strategies; using social media to gain awareness on how the target audience is receiving the campaign and examining the results to help foresee any future uncovered problems that may occur.
Question: Out of all of the evaluation design options, how do you know which one would be the best option for a certain type of program?
Chapter 22:
ReplyDelete1. This chapter talks about why we should evaluate social marketing campaigns
2. A). Formative research is done to start to shape the program strategy and pretest the materials prior to implantation. Formative evaluation is then used using the first four steps of the social marketing process; analysis, strategy development, program and communication design, and pretesting.
B) Evaluation can also have its challenges. One major challenge is asking the wrong questions. Try to avoid the temptation to learn as much about the participants, instead try to get straight to the point and don’t ask 20 irrelevant questions. Ask yourself, “what will I learn from this question, and how will I use the responses?”.
Chapter 23:
1. This chapter talks about the use of evaluation design.
2. A) Individual level indicators are made up of data for particular people who are aggregated and analyzed. The most common method used is surveys .Individuals level indicators measures knowledge, attitudes, beliefs, behaviors, stage of change, and demographics.
B) For small scale social marketing programs the most common evaluation design is to take measurements of the target audience before and after implementation. This method is the least expensive if you are conducting primary research.
Chapter 24:
1. This chapter talks about the different evaluation methods.
2. A) Surveys are the most widely used method to assess the success of a social marketing program. By using quantitative data they can be compared to the same measurements before the campaign was implemented.
B) “You can observe a lot just by watching”- Yogi Berra. Observation is a good technique to use to measure people actual behaviors in certain situations. Observing people in a natural setting without their awareness provides a better indication of what they actually do.
Chapter 25:
1. This chapter talks about how to use feedback to improve your program.
2. A) Having improvements for the future is very important. Summarize the guidelines of what you think went well, what did not, and what could you potentially add in the future. Keep these in a file to have just in case you leave your job there are records of it.
B) Feedback should not wait until the end of the programs. Responding to significant information in timely fashion allows you to improve your program when it counts. If you wait till the end it is too late and you won’t be able to make any improvements/changes.
Question:
How can you evaluate your survey’s to make sure your information is accurate and people weren’t just randomly filling them out?
Chapter 22: Evaluation Basics
ReplyDeleteThesis: This chapter focuses on the importance of evaluating social marketing campaigns.
22A: Formative evaluation answers "what should we do?", process evaluation looks at "how are we doing it?", while summative evaluation determines "did we do it?." The varying formats of these evaluation tactics covers all of the bases and focuses on whether the campaign was a success and where errors might have occurred.
22B: There are challenges that will arise when evaluating the campaign, such as having unrealistic expectations or limited resources. Asking the wrong questions and having technical problems are examples of internal and external issues that could arise.
Chapter 23: Evaluation Design
Thesis: This chapter focuses on how to structure your evaluation process to give the best possible feedback.
23A: Using data from existing records is an effective and time appropriate method. If the same group is used for the pre-test and the post-test, the results can more easily be compared, with any glaring results (either positive or negative) identifiable.
23B: Control groups that haven't been exposed to the campaign will allow for an un-biased reaction, which will yield more reflective results. Evaluation indicators such as behavioral outcomes or policy changes will measure whether the campaign achieved its purpose.
Chapter 24: Evaluation Methods
Thesis: This chapter focuses on the tangible evaluation methods that will be used to determine the success of the social marketing campaign.
24A: Surveys are the most common evaluation method used, and they can be conducted in a variety of ways. Mailed questionnaires and telephone interviews are two forms of surveys that are often used to gather data.
24B: Pure observation isn't as commonly used, because it doesn't yield specific quantitative data, but it can get the job done. There is still a criteria that researchers have to follow when observing the campaign , such as what behaviors to look for and how to code the behavior.
Chapter 25: Using Feedback to Improve Your Program
Thesis: This chapter focuses on using the results from the evaluation process to improve the campaign.
25A: The campaign is only effective if the target audience was effected as hoped, so they're the main focus group of the feedback process. The campaign might not have been perfect but if it can be fixed and improved for the future then it's a success.
25B: Learning what to fix for the future is important because there needs to be specific information given to the organization so they know what worked and what didn't. It's crucial that the organization has effective communication with the target audience to summarize and outline the campaign, looking at both strengths and weaknesses.
Question: How do evaluation methods change based on the type of the campaign? What will the Olympic Committee use to determine the Sochi experience?
Chapter 22
ReplyDelete1.Evaluation will help you identify what does and does not work, so you will be able to focus on your resources on the most effective parts of the program and eliminate or reduce other components.
2.A) The first question to ask when evaluating, “What should we do?” is answered through formative evaluation. Formative research is done to shape the program strategy and pretest the materials prior to implementation. The second question, “How are we doing?” assesses whether the program is executed as planned. Process evaluation determines what information or services were delivered as a result of the program and to whom. The third question, “Did we do it?” involves summative evaluation.
B) An unrealistic expectation can be a challenge you might face in your evaluation. This is because your social marketing program will most likely not dramatically alter these rates in your community overnight. So when evaluating you should be realistic in what you can expect for your results.
Chapter 23
1.When you decide how to structure your evaluation design, this will determine where you find your data, at what points you take measurements, and whether you will use comparison groups
2.A) Use data that are already being collected, either by your organization or by secondary sources, to compare relevant measurements before and after the social marketing program. The advantage of this is so you do not have to start from scratch by conducting new research.
B) The advantage of using individuals as the base of analysis is that you can directly measure changes in the factors your program seeks to influence. However, it can be expensive and time consuming, so community-level indicators might be more appropriate to use.
Chapter 24
1.Collect the evaluation data using a variety of research methods, both qualitative and quantitative to view effects of your social marketing program and areas that need improvements.
2.A) For quantitative, surveys and observation are used. Surveys are the most widely used method to assess the success of a social marketing program and include: mailed questionnaires, telephone interviews, in-person interviews, and computerized surveys. Observation is a technique you can use to measure people’s actual behavior in a given situation or to assess whether they have the skills needed to perform a particular task.
B) For qualitative research, the most useful type of person to recruit as a participant is someone who has been at lease moderately exposed to the campaign messages or materials. This can be done by in-depth interviews, focus groups, or anecdotal feedback mechanisms.
Chapter 25
1.The ultimate purpose of each of your evaluation activities should be to learn how to improve your current and subsequent social marketing programs.
2.A) The feedback function should not wait until the end of the program when summative evaluation activities are complete. One strategy can be to involve key decision makers and stakeholders in the design of program objectives and evaluation planning. Also, keep in mind the technique of backward research to ensure that all the data you collect drive your programmatic decisions rather than sitting around unused.
B) Compile all the documentation from the development and evaluation of the campaign as an appendix for easy reference. Share your results with all those involved in the development of the program including staff, partners, and funders.
Question: When surveying, do you have to stick to one type of survey or can you use several different ones?
Chapter 22: This chapter discusses the importance of evaluating social marketing campaigns.
ReplyDelete1. One of the biggest challenges of evaluation can be asking the wrong questions. Sometimes you get the urge to find out everything you can, instead of focusing on the important questions that are relevant to your study.
2. Through formative evaluation, you should answer the question: “what should we do?” Process evaluation answers: “how are we doing it?” Summative evaluation answers: “did we do it?”
Chapter 23: This chapter discusses how/when to use evaluation design.
1. Using individuals as the base of analysis gives you the chance to directly measure changes in factors. There are downfalls including expense and time, so it can be better to use community-level indicators.
2. It is best to use control groups because they are extremely helpful for comparison. You want to see what happens when someone is and isn’t exposed to see the difference.
Chapter 24: This chapter discusses the different evaluation methods.
1. The most commonly used method is the survey. Quantitative data can be compared using the same measurements before the campaign is implemented.
2. Observation isn’t as commonly used, but it’s simple and an easy way to gain knowledge about a target audience just by watching them.
Chapter 25: This chapter discusses using feedback to improve your program.
1. Real-time feedback is when you improve the program as it goes by implementing the feedback immediately. You should focus on strategy, instead of exploratory formative research.
2. You should not wait until the end for feedback. If you respond to the feedback as your program goes, you can improve it while it counts. You aren’t able to make improvements or changes at the end.
Question: When you’re asking the target audience questions, how do you decide which questions are important and which to leave out?
22. Evaluation creates accountability and is an opportunity to see how effective your campaign was as a whole.
ReplyDelete22.A. “What should we do”, “how are we doing”, and “did we do it” are all questions to examine in evaluation. “Did we do it” is one of the main questions to be answered and it is done through summative research. This means looking at the effects your campaign had on the problem or issue it was designed to address.
22.B. Evaluation can provide great feedback but it can also inaccurately assess a campaign if done poorly. In order for fair evaluation there needs to be realistic expectations, multiple methods in use, and the evaluation needs to start before the campaign is over.
23. A good evaluation design focuses on the places where data will be found, the points that measurements will be taken at, and whether or not comparison groups are necessary.
23.A. Using a comparison group is a great way to evaluate the effectiveness of a message or behavior change. Data already being collected can be researched and used to do this, but testing a group of people (the same group of people) before and after the campaign can provide a viable sample. If this isn’t an option, you can try using questionnaires similar to those of national campaigns and compare trends to apply to your target audience.
23.B. When evaluating individuals things like demographics, stage of change, attitudes, and knowledge can all be used to evaluate however this is time consuming and expensive. Community-based indicators are more cost effective and focus on things like information accessibility, behavioral outcomes, and policy & regulation.
24. Evaluation should use a combination of techniques in order to provide a balance of qualitative and quantitative feedback.
24.A. Surveys and observations are great ways to gather quantitative data. Surveys can administered in person, through the mail, over the phone, and online. This allows a lot of surveys to be done, which provides a lot of data, however it can be limited in its scope.
24.B. In-depth interviews and focus groups provide qualitative data but they are more time consuming and often more expensive to carry out. These methods allow for feedback that isn’t limited by the scope of questions in something like a survey. Individual insight is the real key when using these methods.
25. Feedback is only worthwhile if it is used to improve current and future marketing campaigns.
25.A. Including decision-makers or stakeholders in the process of forming goals and objectives is a way to ensure that feedback will be used. These people will care more about the feedback and want to act on it if they are involved in making the goals – they want to know if the goals they helped create were actually met.
25.B. Regardless of the message, feedback from any campaign can be used to improve that campaign and any future campaign you may work on. Like anything, marketing is something that you get better at the more you do it, and you get better by learning from mistakes in order to create more opportunities going forward.
Question: I guess it depends on what kind of campaign you are working on, but should more stake be placed in quantitative or qualitative feedback when evaluating a campaign?
Chapter 22
ReplyDelete22.) This chapter focuses on evaluating social marketing campaigns.
22a.) Formative research is crucial when shaping a campaigns strategy. Formative evaluation is just as important. Many challenges are associated with this evaluation step including having unrealistic expectations, or using the wrong method completely, which can be disastrous.
22b.) Formative evaluation answers the questions of, “What should we do?” “How are we doing?”, and “Did we do it?”
Chapter 23
23.) This chapter focuses on the use of evaluation design.
23a.) The use of individuals as a base of analysis gives the organization the chance to measure individual factors, and how those individual factors have changed. This analysis is time consuming, so instead of focusing on individual factors, a focus on community-level factors is more beneficial and less expensive.
23b.) Individual factors include demographics, stages of change, and attitudes. Community-based factors include information accessibility, behavioral outcomes, policy, and regulation.
Chapter 24
24. This chapter focuses on different types of evaluation methods.
24a.) Creating surveys is one of the easiest, and most commonly used methods. Surveys include questionnaires, telephone interviews, and computerized surveys.
24b.) Qualitative evaluation techniques help an organization understand the effects and the impacts their campaign has bestowed. Examples of qualitative evaluation methods include in-person interviews, and focus groups.
Chapter 25
25. This chapter focuses on how an organization can use its feedback to improve its campaign in the future.
25a.) The feedback function should never wait until the end of the program. If there is a consistent focus on implementing feedback immediately, a campaign is able to implement changes and improvements during crucial times.
25b.) Focusing on changes immediately is effective, as well has developing a list of improvements for the future of the campaign. As a whole, an organization needs to develop a list of what went well, and what went terribly. Adding to this list in a consistent manner will streamline what has happened in the past--good or bad. This strategy will then create a guideline that the organization can refer to in the present as well as in the future.
Q: Depending on what type of marketing strategy is being implemented, is there an evaluation technique that works better for certain types of campaign strategies?
Chapter 22
ReplyDelete1. Evaluating the different aspects of a social marketing program is very important and there are evaluation challenges that can improve the program but can be difficult.
2. a. There are three different types of evaluation and they focus on what we should do, how are we doing, and was the goal reached.
b. One possible evaluation challenge could be dealing with limited sources, this is a challenge that social marketers need to asses if they wish to successfully carry out their program.
Chapter 23
1. The first step in evaluation is figuring out what design to carry out for the evaluation, there are either cross-sectional or prospective.
2. a. Data collection does not have to be complicated or even new, data for the design can be used from data that is already being collected or that has already been collected.
b. The most effective way to make sure an evaluation is accurate, there should be a control group that has not been exposed to the manipulation of the evaluation design.
Chapter 24
1. After evaluating the design it is time to choose a method to collect the data. There are a few ways to collect data and they differ with every evaluation.
2. a. One way of collecting survey data is by telephone interviews, this type of data collection is great because it increases the response rate because it is a person-to-person interview.
b. Another method that can be used to collect data is observation. This method can be very efficient because people can be observed in their natural environments and this can yield really accurate data.
Chapter 25
1. Feedback is key in every social marketing program that wishes to have an impact on changing behavior.
2. a. Real time feedback means not waiting until a program is over to improve it but rather as the program is coming along.
b. Some things to keep in mind once the social marketing program is completed are which elements of the campaign worked and which did not, and which objectives require additional effort or a new approach.
Question:
How do social marketers decide which survey send? There are so many survey methods, can they all be sent out?
Chapter 22
ReplyDelete1. Evaluation of your campaign can create accountability and can also provide useful information as well as improve your campaign.
2A. There are three questions that you must answer while performing an evaluation. First, "What should we do?", then, "How are we doing?", and lastly, "Did we do it?".
2B. An evaluatin can provide valuable data and feedback, but you may run into evaluation challenges. These can be unrealistic expectations, limited resources, reliance on a single method, using the wrong model, asking the wrong questions, technical problems, resistance from program staff or participants, waiting until the program is over to start the evaluation, and failure to use evaluation results.
Chapter 23
1. First, you must decide how to structure your evaluation design and it will determine where you find data, at what points you take measurements, and whether you will use comparison groups.
2A. You can use data from existing records to compare your data, or can compare with state or national data that will give an indicator of overall trends. Also you can us same-group pretest-posttest, which is taking measurements from the target audience before and after implementation.
2B. You can use a control group that has not been exposed to the campaign for comparison. And lastly, you can use individual-level or community-based indicators to tell you whether you have achieved your social marketing objectives.
Chapter 24
1. Collect the evaluation data using a variety of research methods, both quantitative and qualitative, for the broadcast view of the effect of your social marketing program and areas that need improvement.
2A. One method is surveys, or quantitative data, which can be collected through mailed questionnaires, telephone interviews, in-person interviews, and computerized surveys. You can use quantitative to compare to the same measurements before the campaign was implemented.
2B. Another method is observation, a technique to measure people's actual behavior in a given situation. Lastly, there are qualitative methods, like in-depth interviews, focus groups, and anecdotal feedback mechanisms.
Chapter 25
1. The ultimate purpose of each of your evaluation activities should be to learn how to improve your current and subsequent social marketing.
2A. Responding to relevant information in real time allows you to improve your program when it counts. Effectively using feedback as an interactive, as well as iterative, process in which the elements of your programs constantly are adjusted in relation to other component based on new information.
2B. You can learn and apply the lessons gleaned from the outcomes of this program. Also, think about what you might add, are there new developments in the issue you addressed that should be included in future campaigns?
Question: Should evaluation techniques only be done in the beginning, or can they occur mid-way through to make sure your campaign is still on the right track?
Chapter 22
ReplyDeleteOne must understand why they should do evaluation, what types of evaluation exist, and evaluation challenges, because evaluation is an important part of the process.
-There are three types of evaluation the book identifies: summative, process, and formative. Formative evaluation can help answer the question “what should we do?” Process evaluation helps to answer “how are we doing?” and lastly summative answers “did we do it?”.
-Some evaluation challenges are having unrealistic expectations. A challenge is often being realistic in what you can really expect for the results that you get. For example, an incorrect theory of causation of the problem is a factor that makes this evaluation stage a challenge.
Chapter 23
There are four commonly used approaches in evaluation design and the design chosen will depend on resources as well as length of program (time).
-Using control groups allows you to be certain that the results were based on the effects of the program. To use control groups, you can implement an experimental design or a quasi experimental design.
-An evaluation design shall consist of a few elements. There should be clear program goals and objectives, data to be collected, a methodology, data collection instruments, data processing and analysis, and a evaluation report. The evaluation report helps one to discuss what the key findings of the results were and make some recommendations for possible directions and changes with the program.
Chapter 24
There are a few common evaluation methods, that are usually determined by the design, that are helpful to use during the evaluation stage.
-Surveys are one form of an evaluation method. They can be in the form of a mailed questionnaire, telephone interview, in-person interview, or in the form of a computerized survey. This method is the most common and has some advantages as well as disadvantages, just as any method would.
-Observation is another common evaluation method. This method allows you to measure the behavior of a person in a situation. One of the best ways to carry out observation is in a natural setting because people aren’t aware and can help to eliminate some bias.
Chapter 25
The purpose of evaluation, is to actually take the results/feedback and use it to improve the program.
-In order to make improvements for the future there are a few things to consider changing or simply taking away. Think about what elements actually worked and which ones did not. Also consider what channels were most effective to the program and getting your message out. These can/should be put into the evaluation report.
-Real-time feedback is responding to events as they occur, and not waiting until the complete end of the program to do so. One effective strategy is using social media in order to gage how the audience is responding, what they are saying.
Question:
Sometimes it’s hard to evaluate your own work and progress, are there instances when it helps to bring in a third party to help with the evaluation process?
22.) This chp goes into the evaluation of marketing campaigns.
ReplyDelete1. Summative research answers the question of "Did we do it?" as more of a final result rather than the question of "What should we do?" which is asked earlier.
2. One of the main challenges that is faced would be asking of the wrong questions. Along with trying to fully understand human behavior which is difficult since everyone is different.
23.) This chp explains the evaluation design and its uses.
1. It is much more time efficient to focus on community factors than to narrow the design towards individual factors.
2. Control groups that have not been exposed to any kind of prior manipulation will provide some of the best data results.
24.) Both qualitative and quantitative evaluation methods are very useful for attaining feedback.
1. Surveys are the most efficient means quantitative data collection as an evaluation method.
2. Focus groups and in person interviews are examples of qualitative methods that produce more in depth feedback but are not as efficient as surveys.
25.) This chp talks about how to then use the feedback in ways to improve your program.
1. It is important to act immediately on feedback that you receive and not let it wait in order to make improvements on the go and not wait till the end.
2. It is also important not to let any feedback go unnoticed because any feedback can be used to improve your program.
Q: Is there a certain period of time to wait before trying to collect data and getting feedback? Do certain methods work better for certain ages, genders, ethnicities etc.?
Chapter 22: Evaluation Basics
ReplyDeleteThesis: Evaluation creates accountability, which affects everyone involved in the program and can be helpful to creating a more effective campaign or strategy.
1. “Three simple questions pertaining to each level of evaluation research: What should be we do? How are we doing? Did we do it?” (page 260).
2. Common evaluation challenges include: unrealistic expectations, limited resources, reliance on a single method, using the wrong model, asking the wrong questions, technical problems, resistance from program staff or participants, waiting until the program is over to start evaluation, and failure to use evaluation results.
Chapter 23: Evaluation Design
Thesis: The goal of your final evaluation is to determine the effectiveness of your program on the target audience.
1. For small-scale operations with little to no budget, a common way to accomplish evaluation is to assess the target audience before and after implementation.
2. Individual indicators of effectiveness include: knowledge, attitudes, beliefs, behaviors, stages of change, and demographics.
Chapter 24: Evaluation Methods
Thesis: Quantitative and qualitative research methods should be used for accurate results.
1. Surveys can be administered in a variety of ways: mailed questionnaires, telephone interviews, in person interviews, or computerized surveys.
2. Observation is helpful when measuring a concrete behavior change.
Chapter 25: Using Feedback to Improve Your Program
Thesis: Constructively using feedback to improve your program is a key to its success.
1. Real-time feedback on social media is an opportunity to answer questions from the target audience, clarify misconceptions, and address complaints.
2. “In addition to considering the elements you would change or eliminate from your program, think about what you might add,” (page 278).
Question: Should there be a specific group of people within the campaign whose job is to evaluate the program’s success throughout the whole process?
Chapter 22
ReplyDelete1. Evaluation is beneficial to social marketers and creates accountability though several different forms and subsequent challenges.
2a. Formative evaluation shapes program strategy and pretesting materials before implementation. Process evaluation addresses the question of “how are we doing?” to discover whether or not the program is being executed as planned. Summative evaluation uncovers the effects your program had on the issue it was targeting.
2b. Challenges can arise when social marketers have limited resources and experience resistance from program staff or participants. It is often that social marketers have enough liquid cash to fund their program itself, without the evaluation piece. Staff can feel threatened by the thought of being “evaluated” and members of the target audience simply do not see the point or benefit to giving up information about their personal lives.
Chapter 23
1. Evaluation design should be structured to increase chances of discovering if it was in fact your campaign and not outside influences that caused any differences though various approaches.
2a. It is possible to use data from existing records. This data could have been collected previously by your organization or secondary sources and can be used to compare relevant findings before and after the campaign.
2b. Changes in evaluation indicators can tell you if your social marketing objectives have been reached. These indicators occur at either the individual or community level and include beliefs and demographics on the individual level as well as policy and regulation and information accessibility on the community level.
Chapter 24
1. Evaluation data should be collected from both quantitative and qualitative methods for the most inclusive view of the effects of your social marketing campaign.
2a. Surveys are a way of obtaining quantitative data. They are the most widely used method of data evaluation and include telephone interviews, in-person interviews, and computerized surveys.
2b. Qualitative methods add a different perspective on the effects of your campaign. They aim to draw our more detail and context. Methods include focus groups and anecdotal feedback mechanisms.
Chapter 25
1. Evaluation activities should ultimately provide insight as to how to improve current programs and shape subsequent marketing programs.
2a. Feedback should not wait until the end of the program. It should be gathered in real-time to best address problems with your program when they take place.
2b. Lessons learned from the program can be applied to future programs. Social markets should consider which distribution channels were most effective as well as how best to use their budget.
Question: Are there studies which show which evaluation technique might be the best for a certain demographic?
22. Even though the evaluation takes place after the campaign, it is a vital piece of the campaign.
ReplyDeletea. Evaluation is important because it creates accountability. This is an opportunity for many within the company to work hard and prove that they were a valuable piece in the success of the campaign.
b. There are many challenges in performing evaluations. These include unrealistic expectations, limited resources, reliance on a single method, using the wrong model, asking the wrong questions, technical problems, resistance from program staff or participants, waiting until the program is over to start evaluation, and failure to use evaluation results. All of these challenges can create a poor evaluation process which can make it so a lot of well done work goes unnoticed.
23. Evaluation design is important to consider and is dependent on what kind of campaign you conducted.
a. There are many elements involved in evaluation design including methodology and data collection instruments such as the survey questionnaire.
b. You can measure changes in the factors your program seeks to influence directly if you use individuals as the base of analysis.
24. There are 3 methods to evaluating a campaign which are surveys, observations and qualitative methods.
a. Qualitiative methods cannot be validly compared to the results of qualitative research before the campaign but the data gathered using these methods will give a good indication of which elements were effective from the target audience's perspective and how the campaign could be improved for the next phase.
b. In person interviews are a great but can take up a significant amount of time where computerized surveys might be more time effective.
25. After you conduct evaluations you can use this information to improve your program for the next phase.
a. It would be best if evaluation was done throughout the program and you can always use feedback as a way to conduct a better program. Responding to relevant information in real time is much more valuable than retrospectively.
b. In an evaluation report you should include the program background, evaluation methodology, evaluation findings, and recommendations.
question:
If a campaign is ended such as VERB, where funding was cut, what would be the purpose of finishing evaluation efforts?
Chapter 22 - Evaluation is an important step because it creates accountability, and allows you to improve your program while it is being implemented, or make necessary changes for future endeavors.
ReplyDeleteA. Impact evaluation really tells you if your social marketing program worked – if it had an impact. It connects the dots between behavior change and health/social outcomes. Impact evaluation determines whether or not the consumers who adopted your behavior experienced a decrease in morbidity or mortality.
B. One specific challenge of the evaluation process is failing to use your evaluation results. Many evaluations end up on bookshelves or filed away rather than being used as a recommendation for the next steps of your campaign. An evaluation should be a living document rather than an old dusty book.
Chapter 23 - The evaluation design that you choose will determine where to find data, at what points measurements can be taken, and whether or not comparison groups will be used. This step should be completed before you even choose a research method.
A. Using a control group is usually the most accurate method of evaluation, however it is the most complex and can be the most costly. It involves evaluating a control group who has not been exposed to your campaign, and comparing this to your target audience who has been exposed.
B. Community based indicators of achievement in an evaluation process include environmental change, policy/regulation, information accessibility, and behavioral outcomes.
Chapter 24 - For the broadest view and deepest understanding of your social marketing campaign, use both quantitative and qualitative evaluation research methods.
A. Using observation as an evaluation method allows you to measure people’s actual behavior in a given situation to determine whether or not they have the necessary skills to perform a particular task.
B. Qualitative evaluation methods involve slightly different questions than something you would use in a survey – they involve more detail and context. Some examples include: What do you like/dislike about the campaign? Do you think many of your peers are aware of the campaign? How could we reach more people like you?
Chapter 25 - The ultimate goal of evaluation is to learn how you can improve your current and future social marketing programs.
A. One strategy to help your results be more practical and “real-time” actionable is to use social media to research what your audience is saying about your campaign while it’s running – this way you can make changes if necessary.
B. Some lessons evaluation may teach us about campaigns are: which elements of the campaign worked and which did not, how to make the social marketing process run more smoothly, and what types of challenges to anticipate next time (and how to overcome them!)
QUESTION: How does the evaluation process differ when used outside of social marketing campaigns? (For example, in traditional marketing?)
Chapter 22
ReplyDelete1. Evaluation helps to understand how effective your marketing strategy was. It can prove that your plan has made a difference, which can secure more funds for you in future social marketing endeavors. Evaluation also shows you areas that could be strengthened or possible mistakes that you have made.
2A. Ethical evaluation is one form of evaluation. Ethical issues should be considered at each stage of program development to ensure the integrity of target audience research and that the program itself does not encourage behavior change in an unethical way.
2B. When considering evaluations, don't have unrealistic expectations. Your campaign isn't going to impact change over night. It might take a couple years to develop. Understand the attitudes, demographic, awareness and knowledge of your target audience in order to illicit change.
Chapter 23
1. Evaluation design will determine where you will get you data from, at what points you take measures, and if comparison groups will be necessary.
2A.Collecting data from existing records is very beneficial to your overall design. Using data already collected from your organizations or by secondary sources compares relevant measurements before and after your social media marketing experiment.
2B. There are several different kinds of data collecting instruments. Survey questionnaires, focus groups topic guide, along with other research instruments should be developed and pretested prior to its use in a real evaluation.
Chapter 24
1. Evaluation methods whether they are qualitative or quantitative, can offer the the broadest view of how effective your social marketing program is, while also bolstering areas that need improvement.
2A. Surveys are the most common method to assess the success of a social marketing program. Some examples include mailed questionnaires, telephone interviews, in-person interviews, and computerized surveys.
2B. Qualitative methods can help you explore your research while planning your program, which helps you better understand the effects of you campaign. Some examples are in-depth interviews, focus groups, and anecdotal feedback.
Chapter 25
1. Using feedback to improve your program should help you learn how to improve your current and subsequent social marketing programs. This will also identify other areas that need to be strengthened and also recognize glaring mistakes.
2A. Feedback is important for future improvements by revealing objectives that need more attention or a different approach and how to realistically budget your time and funds for the next phase.
2B. Real time feedback is key. Feed back should not wait till the end of a program. Responding to relevant information in real time allows you to improve your program when it counts rather than in hindsight.
Question: Does feedback have take a toll on the overall budget by revealing mistakes that need to be adjusted or are possible mistakes factored in prior to the launch?
Chapter 22
ReplyDelete1. This chapter discuses the importance of and why we should evaluate social marketing campaigns.
2a. Funds or money can be based from evaluation measures. Can be something that could potentially benefit you if your evaluations are good.
2b. There can also be challenges associated with an evaluation. Ensure you have realistic expectations and are working within the resources you have.
Chapter 23
1. This chapter discussed how to go about constructing and designing an evaluation.
2a. The structure and design of your evaluation is extremely important you need to be able to identity where and how you will find or collect the data and how you are going to measure it.
2b. Make sure you use the resources available to you sometimes there may be current questionnaires available to you.
Chapter 24
1. This chapter discussed the method you will use to collect the evaluation.
2a. Discussing which collection method is very important there are many ways. Surveying being one of the most popular.
2b. Simply watching and observing is a very beneficial way to collect or have someone collect and evaluate.
Chapter 25
1. This final chapter discussed how to use the information and feedback to help enhance or improve the project you are working on.
2a. At times the feedback may be received to late for your current project but could be helpful tips for future projects or campaigns.
2b. You want to ensure you have learned from mistakes made and getting real life feedback is one of the best ways to gage your success.
Question:
When the information collected and evaluated isn't used to help the current project and future projects or marketing plans are intended was there time waste? Or time that could have been better spent elsewhere to make the project more successful the first time around?
Chapter 22:
ReplyDelete1. Evaluation of a certain campaign in a vital piece of a well-planned social marketing program, and although daunting, it should not be ignored.
2. The benefits of evaluation are numerous. Evaluation creates accountability, it will help you improve your program, and it provides useful information, not just statistics. If done properly, the three simple questions- what should we do? How are we doing? Did we do it? - will all be answered. Formative research occurs prior to implementation, process evaluation occurs during implementation, and summative evaluation occurs post-implementation, and these three types of evaluation answer the three simple questions in respective order.
Chapter 23:
1. The structure of your evaluation will determine where you find your data, at what points you take measurements, and whether you will use comparison groups, and what structure you choose depends on your resources and amount of time given to the implementation of your program
2. In order to evaluate effectively, it is important to look at the evaluation indicators of knowledge measures, attitude measures, measures of beliefs, behaviors, stage of change, and demographics. Also important to look at in evaluation, include environmental changes, policies and regulations, information accessibility, and behavioral outcomes. All of these are indications of how successful your program was in promoting behavioral change or knowledge of the issue.
Chapter 24:
1. Depending on the indicators that are selected, and which evaluation design chosen, the method of evaluation will differ.
2. Common research methods used in evaluation include surveys (mailed questionnaires, telephone interviews, in-person interviews, and computerized surveys), observations, and qualitative methods including focus groups, in-depth interviews, and anecdotal feedback mechanisms. The questions you can ask in a qualitative method type evaluation will differ than those that you may ask in a survey, since they can draw out more detail and context.
Chapter 25:
1. The end goal with evaluation is to identify how to improve the campaign and make those necessary changes so that the evaluation process does not go to waste.
2. Use feedback during your program, focusing on real-time feedback, in order to respond to relevant information in real time, or as the events occur. This will help you to effectively use feedback to adjust your campaign. Also, when evaluating, consider the improvements that can be made for the future, such as locating effective channels or correctly budgeting time and money.
Question:
Are there any stand out campaigns that have made drastic changes to their program based on evaluation that you can give us an example of?
Chapter 22 – “Evaluation Basics”
ReplyDeleteThesis: Evaluation is important because it creates accountability, improves your program while being implemented, and can prove the success of your program.
Support 1: There are four types of evaluation. They are formative evaluation, process evaluation, summative evaluation, and impact evaluation.
Support 2: One of the challenges associated with evaluation is having limited resources. However, there are ways to have a good evaluation strategy at little to no cost. One way is to team up with a university researcher who is interested in the same subject and will be willing to do the evaluation themselves.
Chapter 23 – “Evaluation Design”
Thesis: When designing an evaluation strategy, consider your resources, the duration of the program, and your expertise.
Support 1: A common evaluation design involves taking measurements of the target audience before and after the program. Comparing the results can show a picture of the success or failure of the program.
Support 2: It is important to rule out other factors when determining the success of your program. This can be done through the use of control groups. The control group should be about the same as your target audience with the sole exception that they have not seen the campaign.
Chapter 24 – “Evaluation Methods”
Thesis: Choose an evaluation method based on your evaluation design and use it to collect quantitative and qualitative data.
Support 1: You can use surveys to collect data. Mailed questionnaires can be easy to carry out but often have a low return rate. In-person interviews can help you get a lot of information from each interviewee, but there is much more effort involved and you usually cannot speak to as many people as in other surveying methods.
Support 2: Observation can be a useful methods for collecting data because it eliminates the bias people may have for themselves like when they are self-reflecting in a survey. Observation can also show whether the call to action in the campaign is being carried out correctly by the audience.
Chapter 25 – “Using Feedback to Improve Your Program”
Thesis: The goal of evaluating your campaign is to identify points of improvement based on feedback.
Support 1: It is ideal to be receiving feedback in real time. This allows you to respond immediately with changes to make your campaign most effective.
Support 2: Feedback is also helpful for making improvements in the future. Identify ways to make the program run more smoothly, work more effectively, and be budgeted realistically.
Question: Assuming you will not be doing the same type of campaign more than once, is there a way to evaluate your program so that your data can be useful in future unrelated programs?
Chapter 22. The chapter discussed the steps to evaluate a social marketing campaign.
ReplyDeletea.) Three question that included should be always be answered in all well planned social marketing campaigns are; What should we do? How are we doing? and Did we do it?
b.) There are several challenges that will come up when administering an evaluation such as having unrealistic expectations, limited resources, and reliance on (1) method.
Chapter 23.The chapter was about evaluation design.
a.) The use of control groups will aid in evaluating the true effect of the social marketing campaign
b.) The use of surveys is the most common method for collecting individual level data for analysis it validate if your campaign is meeting its outcome objectives.
Chapter 24.The chapter discussed different evaluation methods
a.) The use of in person interviews and computerized surveys are the most common method for capturing data and feedback to evaluate your campaign.
b.) The most common qualitative method for capturing data is also through the use of in-depth interviews, focus groups and antidotal feedback.
Chapter 25.This chapter talks about the Use of Feedback to Improve Your Program
a.) Real time feedback to improve your program allows you to improve your program at a time when it counts as opposed to at the end when it is too late to make changes.
b.) Three strategies to help make your results practical and useful are the use of preliminary evaluations, be flexible to change and use summary feedback for use in future program decisions.
Question: Is there recommendation on How far along should you go before using real-time feedback to improve your program? Daily, Weekly, Half-way?
Ch.22
ReplyDeleteA) Here we are taught why to evaluate our social marketing campaigns.
B)
"The first question "What should we do?" is answered through formative evaluation. Formative research is done to shape the program strategy and pretest the materials prior to implementation. "
"Evaluation creates accountability."
Ch.23
A) This chapter is all about how we should design our evaluation
B)
" Use data that are already being collected, either by your organization or
by secondary sources, to compare relevant measurements before and after the social marketing program."
"The advantage of using existing data is clear: You do not have to start from scratch by conducting new research."
Ch.24
A) All about different methods of evaluation
B)
"Surveys are by far the most widely used method to assess the success of a social marketing program. Quantitative data, preferably those that can be compared to the same measurements before the campaign was implemented, are the only way in which to really demonstrate behavior change and other effects to funders or critics of your program."
" Although you generally cannot validly compare the results of qualitative research before and after the campaign, the data gathered using these methods will give a good indication of which elements were effective from the target audience's perspective and how the campaign could be improved for the next phase."
25.
A) this chapter is all about showing us how to use the feedback we got from this evaluation.
B) "Responding to relevant information in real time— as events
actually occur— allows you to improve your program when it counts rather than retrospectively realizing what you should have done."
". Effectively using feedback is an interactive, as well as iterative, process in which the elements of your program constantly are adjusted in relation to other components based on new information"
Q: When can the evaluation process start?
Chapter 22
ReplyDelete1. Evaluation is an integral part of creating a campaign because it creates accountability. This allows you to see what about your campaign works and what doesn't but this also makes wary of having their programs evaluated because it could show that some people are not working to their full potential.
2.a. Lack of resources can be one of the largest barriers to conducting an evaluation. Most of the tie social marketing projects have just enough money to conduct their campaign and there really isn't funding to perform an evaluation.
B. Evaluations can also be met with resistance by the program's staff. Staff are usually not the biggest fans of evaluations because it means that the quality of work that they are doing is being called into question.
Chapter 23
1. It is very important to properly structure the design of your evaluation. This will depend on how much time you have to evaluate and the amount of money you have to perform it and you evaluation will either be cross-sectional, or prospective.
2.a. When you are creating a social marketing campaign it is important to be sure that you can actually see the effect your campaign is having on your target audience. You can do this by having a control group to reference that has no exposure to your campaign.
B. To tell whether you have achieved your social marketing objectives it is important to look at evaluation indicators. These will show you where your program has been successful and where it needs to be worked on.
Chapter 24
1. Through the use of both quantitative and qualitative research methods you can see the effects of you social marketing program and the areas that need work.
2.a. You can use a technique called observation to measure the behaviors of people in a given situation. This can also be used to see what skills people need to have in order to perform a particular task.
b. In order to lean about the context in which the audience received your message it is important for a trained interviewer to meet one on one with the participants.
25. Feedback is a huge part of making a successful social marketing campaign. The results you gather from feedback can be used to make improvements on your campaign.
2a. One strategy for making your evaluation results more applicable is by using backward research to make sure that all your data moves your programmatic decisions rather than sitting around unused.
b. Another strategy to make your evaluation results more practical is by pretesting results to build a strong communication strategy.
Question: How effective is the use of pretesting if you use a the same group of people for both the pretest and post test and do not use a control group?
Chapter 22:
ReplyDelete1. This chapter focuses on the importance of evaluating social marketing campaigns.
2.a. Evaluation focuses on the question " Did the program work?" This question is part of formative evaluation, the first of three components. Followed by, process evaluation, and summative evaluation.
b. Challenges that may arise when evaluating the campaign may include unrealistic expectations, having limited resources, or relying on just a single method.
Chapter 23:
1. Structuring your evaluation design determines where your data will be found, how to measure the target audience, setting a standard, and whether or not control groups are necessary.
2.a. It is best to use data from your organization that is currently being collected to compare relevant measurements. If the data answers your evaluation questions, you don't need to start from scratch.
b. Small scale programs may want to measure their target audience before and after implementation. Conducting primary research is a good way to collect valid data.
Chapter 24:
1. There are a variety of qualitative and quantitative methods to collect data upon evaluation.
2.a. Surveys are a popular way to collect quantitative data. They measure the success of the campaign by comparing data from before and after the campaign implementation.
b. Qualitative methods may include interviews, focus groups, or feedback mechanisms. Anecdotal feedback mechanisms share the stories of individuals reflecting the campaign on a personal level.
Chapter 25:
1. The goal of evaluation is to determine how to improve your social marketing program.
2.a. Feedback should be used in real-time, making changes to the program as events occur. This is an interactive process where you are continually making adjustments.
b. Whether the campaign will continue or a new one will be implemented, there are a few things to reflect on. In the future, you might focus on how to make the process run smoother, what worked and what didn't, what channels were most effective, and what challenges to anticipate.
Question: How do you know how many evaluation methods to use, and should every campaign evaluate throughout the whole process?
Chapter 22:
ReplyDeleteThesis- Evaluation is the final step of a successful social marketing campaign: it provides useful information and creates accountability.
•“Evaluation creates accountability. For this reason, many people are wary of having their programs evaluated and would to prefer to rely on anecdotal evidence of success rather then actual data that may or may not reflect well on them.” (p 259)
•“Balch and Sutton aptly pose three simple questions pertaining to each level of evaluation research: what should we do? How are we doing? Did we do it?” (p 260)
Chapter 23:
Thesis- Having an evaluation design will enable a social media campaign to make an accurate and constructive conclusion with the information gathered from their evaluation.
•“The design you use to carry out the evaluation will depend in part of your resources and the length of time the program is implemented. An evaluation design can be either cross-section (i.e., administered to people randomly chosen at particular points in time) or prospective (i.e., following the same individuals over time).” (p 265)
•"Before you decide the actual method you will use, determine the points at which you will collect the data and how complex the evaluation design will be."(p 265)
Chapter 24:
Thesis- Having a variety of evaluation data (qualitative and quantitative) enables social marketers to get a broad view of the effects of their campaign as well as where it needs improvement.
•“The methods you use will be determined in part by the evaluation design and indicators you select.” (p 271)
•“Surveys are by far the most widely used method to assess the success of a social marketing program. Quantitative data, preferably those that can be compared to the same measurement before the campaign was implemented, are the only way in which to really demonstrate behavior change and other effects to funders or critics of your program.” (p 271)
Chapter 25:
Thesis- Feedback is the most important way to improve a social marketing campaign: it allows you to adjust the current program as well as improve the future strategy.
•“The ultimate purpose of each of your evaluation activities should be to learn how to improve your current and subsequent social marketing programs. The results will do no good to any one if they are not acted on or are immediately retired to a shelf.” (p 277)
•“Effectively using feedback is an interactive, as well as iterative, process in which the elements of your program constantly are adjusted in relation to other components based on new information.” (p 277)
Question: Is a social marketing campaign ever end? It seems you can always make improvements until you run out of funding to do so.
Chapter 22
ReplyDelete1. Evaluation is key to establishing accountability and reliability in a social marketing platform.
2. A. To evaluate your marketing program, you must ask yourself- “What should we do”, “How are we doing,” and “Did we do it.”
B. Having unrealistic expectations can become a problem in the evaluation process, especially when encountering the “did we do it” question.
Chapter 23
1. There are specific and different evaluation designs, suited for different occasions.
2. A. One way in which evaluation can be done is by comparing results of the marketing campaign to existing data about the demographic.
B. It is important to use control groups to compare your data to. They must have no contact or exposure to the campaign.
Chapter 24
1. There are multiple ways of collecting data for evaluation, some of which are quantitative while some are qualitative.
2. A. Surveys are a useful method for evaluation if you are trying to gather information from a large sample without spending a lot of time and money.
B. Observation is a method of qualitative evaluation that is useful, but often needs to be applied with quantitative methods as well.
Chapter 25
1. Feedback is a constructive tool that must be used to improve you campaign going forward.
2. A. Focus groups are an invaluable tool when it comes to finding out how your target audience views your campaign.
B. Even if your campaign is over, information gathered through feedback can be used for future, similar campaigns.
Question-
Is there a general rule of thumb towards how much negative feedback must be received in order to change the campaign, based on the size of the campaign itself?
Chapter 22—
ReplyDelete1. Evaluation is important because it creates accountability, and if done well, your evaluation activities will help to improve your program while it is being implemented as well as in later incarnations.
2.A. When you dive into your evaluation you should primarily focus on the question “did the program work?”. However, evaluation requires more than the answer of this broad question alone. Rather, also pose the following questions to your research: What should we do? How are we doing? Did we do it?
B. Done correctly and evaluation can provide valuable feedback and data. Done poorly, and an ineffective campaign might be reviewed well, or well put together program might go completely under the radar.
Chapter 23—
1. When it comes to your evaluation design, the one you choose will all depend on the resources you have available as well as the length of time your program is implemented.
2.A. When you’re working with small-scale marketing programs, the most common evaluation design is to take measurements of the target audience before and after implementation. This method is the least expensive for collecting data if you are conducting primary research.
B. Let it be known, that if your target audience is significantly different in key ways from the population used to determine the standard, then this design will not bring in accurate results.
Chapter 24—
1. In order to see the effects of your social marketing program, one should collect both quantitative and qualitative data, amongst other research methods.
2.A. On the more qualitative side, it’s important to realize how far simply observation can go. It’s a technique you can use to measure people’s actual behavior in a given situation or to assess whether they have the skills needed to perform a particular task.
B. The data gathered using qualitative methods will give a good indication of which elements were effective from the target audience’s perspective and how the campaign could be improved for the next phase.
Chapter 25—
1. The end goal of your evaluation activities is to learn how to improve your current social marketing programs, in order to make them more effective in the future.
2.A. When you respond to relevant information in real time, when events actually occur, it allows you to improve your program when it counts rather than retrospectively realizing what you should have done.
B. Lessons like “how to make the social marketing process run more smoothly” or “which elements of the campaign worked and which did not” will help you continue with additional phases of implementation of the same campaign or design future social marketing programs on other topics
Question—Who holds the responsibility of managing the negative and positive feedback of a campaign? Is there a separate team for that task?
Chapter 22
ReplyDeleteThis chapter suggest that you should see evaluation as a opportunity for you to improve the program by answer three simple questions pertaining to each level of evaluation research : What should we do? , How are we doing? And Did we do it?
- The question “Did we do it” involves “summative evaluation”. Summative evaluation includes outcome evaluation and impact evaluation. Our textbook suggests that we should identify the extent of attitude and behavior change in the target audience campaign and correlating it with individual exposure to the campaign in the outcome evaluation step.
-Ethical evaluation suggests the idea that People should be involved in the program for their own good, not to be coercive in a behavior. Using money as an incentive may be effective in the short term, but not effective in the long term.
-
Chapter 23
This chapter illustrates different elements of an evaluation design.
- The true experimenter design allows you to identify the inside elements in your campaign that cause change, rather than outside elements. Random assignment a people and community either to participate in the program or not will help you eliminate the barriers.
- Using collected data by different organization is also a good idea. These measurements could be statistics your program collects routinely. You can use the data to compare the behavior change before and after the campaign.
Chapter 24
This chapter differentiates the use of different research methods.
- Observation is the technique you can use to measure people`s actual behavior in a given situation. Observation requires a systematic approach to ensure valid research result. You need to create a protocol for the observers to follow that tells them where and when to conduct the research.
- Anecdotal feedback mechanism suggests that the stories of individual people can be quite powerful as a way of demonstrating the program`s effects as a personal level. This can help ”bring life” to audience behind the statistics.
Chapter 25
This chapter introduces the method to use feedback to improve our campaigns.
- Feedback should be responded to relevant information in real time, as the actual occurring of the event. The process of using feedback needs to be interactive, iterative and adjusted to other components based on new information.
- You can use the outcome of this campaign to the future campaign. You should summarize the guidelines in a written evaluation report so that a record will exists for other to read in the future.
Questions:
Can we really trust data that collected by different organization ( in term that their data based on different population compared to our campaign)
Chapter 22
ReplyDelete1. The final step of a social marketing program is evaluation. It is a crucial step and increases accountability.
2. “Baltch and Sutton aptly pose three simple questions pertaining to each level of evaluation research: What should we do? How are we doing? Did we do it?” (260)
3. “If done well, you evaluation activities will also help improve your program while it is being implemented as well as later incarnations.” (259)
Chapter 23
1. Using an evaluation design with data from existing records, same-group pretesting-posttest, comparison to standard, and evaluation indictors can help determine the effect of your program on the target audience.
2. “This will determine where you find your data, at what points you take measurements and whether you will use comparison groups. The design you use to carry out the evaluation will depend in art of your research and length of time the program in implemented.” (265)
3. “Because true experimental research can be very costly and requires a huge level of expertise, it is not practical to expect from small-scale marketing program. You can, however, structure your evaluation to increase y our chances of obtaining valid results even if you do not use the most rigorous designs.” (265)
Chapter 24
1. The evaluation methods you use will be determined on your evaluation design and the indicators you select, but include both quantitative and qualitative research methods.
2. “Quantitative data, preferably those that can be compared to the same measurements before the campaign was implemented, are the only way in which to really demonstrate behavior change and other effects to funder or critics of your program.” (271)
3. “Although you generally cannot validly compare the results of qualitative research before and after the campaign, the data gathered using these methods will give a good indication of which elements were effective from the target audiences perspective and how the campaign could be improved for the next phase.” (274)
Chapter 25
1. Feedback helps you see the ups and downs of a given campaign and helps you improve on future programs.
2. “Ideally, the feedback function should not wait until the end of the program when summative evaluation activities are complete. Responding to relevant information in real time- as events actually occur- allows you to improve your program when it counts rather than retrospectively realizing what you should have done.” (277)
3. “The following information should be included in your evaluation report: Program background, evaluation methodology, evaluating findings, and recommendations.” (278)
QUESTION: Is your marketing strategy ever completely finished? Or does it constantly evolve as you progress.
Chapter 22: Evaluation Basics
ReplyDeleteThesis: Evaluating your program is an indispensible part of the campaign process, and by determining levels of success, you are able to build accountability and provide information that will help you improve your program.
Point 1: Rather than focusing on the broad question, “Did the program work?” Balch and Sutton suggest three stages of evaluation. The first, formative evaluation, answers the question “What should we do?” Next, process evaluation, tells you what information and services were delivered as a result of your program. Finally, summative evaluation, determines the effectiveness of your program.
Point 2: Poorly conducted evaluations can result in the celebration of an unsuccessful campaign, or an effective campaign might be missed. Stretching funds in creative ways, using several evaluation methods, asking the right questions, seeking advice from expert evaluators, and starting evaluations early are all ways to avoid undesirable evaluations.
Chapter 23: Evaluation Design
Thesis: Figuring out how to structure your evaluation will help answer the questions of where to find your data, at what points you will take data, and whether comparison groups are necessary, and will be determined by resources and time.
Point 1: Using existing data can be an effective tool during the evaluation process, in part because you do not have to start from scratch. In small-scale programs, pretest-posttest methods provide a cost-effective way of collecting valid data. The downside to both of these campaigns is that you can’t necessarily prove a causal effect between your campaign and the changes you note.
Point 2: In order to gather data, you must identify personal and community indicators. Individual indicators include things like knowledge, attitudes, beliefs, behaviors, stage of change, and demographics. Community-based indicators encompass environmental changes, policy and regulation changes, information accessibility, and behavior outcomes. Changes in these indicators will help you to determine the effectiveness of your campaign
Chapter 24: Evaluation Methods
Thesis: In using quantitative and qualitative data for evaluation, you will gain the broadest view of the effectiveness of your campaign
Point 1: Quantitative data is the only way to demonstrate behavior change. Quantitative data usually involves surveys, and often involve comparisons to pre-implementation data. Different kinds of surveys can be used including, mailed questionnaires, telephone interviews, in-person interviews, and computerized surveys. Qualitative research can be a good way to determine the parts of your campaign that were both, effective and ineffective.
Point 2: Observation is a good technique to use in evaluation, but requires a systematic approach that guarantees valid results. Observing people in a natural setting allows you to avoid the risk of getting answers from respondents that don’t necessarily reflect reality.
Chapter 25: Using Feedback to Improve your Program
Thesis: The ultimate purpose of the evaluation process is to determine how you can improve your campaign in the future.
Point 1: Making evaluations throughout the process allows you to correct mistakes during implementation, not just realize at the end what could have been better. It is an interactive process that allows you to make adjustments as information is received.
Point 2: The research you gather can be used in a number of ways to help you improve your campaign during subsequent stages of implementation, or in developing future programs. It can be helpful to summarize your findings in some kind of report that can be drawn upon in the creation of new projects. The report should be “no longer than necessary, easy to read, and provid[e] clear action items” (278).
Chapter 22 discuses the importance of evaluating social marketing campaigns.
ReplyDeleteA common way to evaluate the campaign of a small-scale operation with a small budget is to assess the target audience before and after the implementation.
Failing to use your evaluation results is crucial to the evaluation process. Evaluations should be used as a living document to aid the project, and future projects.
Chapter 23 discusses how to construct and designing an evaluation.
Use resources currently available to you, there may be current questionnaires with helpful information for the evaluation.
Measure the target audience before and after the program. The comparison of the results will show a picture of the success or failure of the program.
Chapter 24 discusses how to choose an evaluation method based on your evaluation design, how to use it, and collect quantitative and qualitative data.
Control groups help to evaluate the true effect of the social marketing campaign.
Observation’s can be made to measure the behaviors of people in a planed situation. From this you can see what skills are needed in order to perform a particular task.
Chapter 25 discusses the goal of evaluation: to determine how to improve your social marketing program.
After your campaign, the information gathered with feedback can be used for future, similar campaigns.
Evaluating throughout your campaign allows you to correct mistakes during implementation, not rather than after the fact. This interactive process allows you to make adjustments as information is understood.
QUESTION: When planning a marketing strategy, how do you know when your research is done? Is it ever complete?
Chapter 22
ReplyDeleteThe author asserts how essential good evaluation can be to a project's success.
A.) Evaluation shows accountability and can be used to show how the project made a difference.
B.) Strict, regimented research can have a place in social marketing evaluation but there are other simpler ways to evaluate success. Three simple questions that guide the process are- What should we do? How are we doing it? And did we do it?
Chapter 23
The focus of the chapter is how to design your evaluation to suit the needs of the project.
A.) "...The ultimate goal is to determine the effects of your program on the target audience."
B.) Indicators are the measurement used to ascertain whether we have been successful in achieving our key outcomes.
Chapter 24
The chapter distinguishes between various evaluation methods, offering information on how best to structure the evaluation to gain the information you need.
A.) The three most commonly used means of evaluating are surveys, observation and qualitative methods.
B.) Qualitative methods can include in-depth interviews, focus groups mind anecdotal feedback methods.
Chapter 25
The final chapter focuses on using the evaluative feedback to strengthen the product and promotion of the social marketing campaign.
A.) One of the first suggestions for making practical improvements is to "Involve key decision makers and stakeholders in the design of program objectives and evaluation planning."
B.) The author also offers that social media can be a great resource in uncovering how people are interacting with your social marketing campaign.
Question: How interested were the patrons in the case study in the promotional non-alcoholic beverage? I.e. How many were handed out and how many were validated?
CHAPTER22
ReplyDelete(1) Evaluations are key in building a reputation of accountability and building accessible means to identify the DOs and DON’Ts as a program advances.
1- If you are able to understand what people do and do not like about your program, you will be more efficient when trying to create a lasting message.
2- Evaluation creates accountability in social marketing and should be seen as an opportunity to show what your program has to offer and the progress it has made.
CHAPTER23
(1) By structuring your evaluation, you will determine where your information comes from, and also determine the different ways to measure results and possible comparisons.
1- While it is important to understand existing data, it is just as important to take what you already have and compile it with your researched and predicted information.
2- It is important to understand your target audience beforehand, so that you know how to read your results in the most effective way possible.
CHAPTER24
(1) Evaluation methods look into both qualitative and quantitative types of research to get the best all-around results for your social marketing program, which ultimately factor the evaluation design and indicators selected.
1- Your results can be evaluated through a variety of different ways; you can compile your results through surveys, observations and consumer feedback.
2- Observing people is important when trying to get accurate, yet objective information. Through observations, you can differentiate what is an actual response as opposed to one that is made up.
CHAPTER25
(1) Feedback is only beneficial if you take it and apply the results to your social marketing strategy. Evaluated feedback can help make progress in any social marketing plan.
1- Avoiding procrastination and making changes as needed is much more efficient than waiting around until deadlines, and is going to reduce potential problems.
2- Being flexible is important when reviewing feedback because you want to use this as a way of improving your program, and what you are taking away from this evaluation/feedback will affect your future as a program.
Question: As a consumer, what is the most important thing to remember when giving feedback to a company or program?
Chapter 22:
ReplyDelete1. The main point of chapter two is to review evaluation basics and understand why they are so important to the success of a social marketing program.
2. A) Evaluations all revolve around the primary question, “Did the program work?” but there are many types of evaluations used to help you assess aspects of your program. For example, impact evaluation determines whether people experience improvement in quality of life related to the overall goal of the program.
B) There are a number of challenges when conducting an evaluation. First, you could have unrealistic expectations in how effective your program is in combating whatever it is you want to change.
Chapter 23:
1. Chapter 23 is about how to structure your evaluation design, specifically whether it is cross-sectional or prospective in addition to a summative goal.
2. A) In order to understand whether you achieved your objectives, you will see changes in evaluation indicators. Indicators include measures such as knowledge, attitudes, beliefs, behaviors, stage of change and demographics.
B) By using control groups in your evaluation, you can create a “quasi-experimental” design. This means your control group could be a community that is similar to your target audience but not exposed to the campaign. That way, you can determine the different effects your program has on the audience.
Chapter 24:
1. Chapter 24 reviews the different evaluation methods such as surveys, observation, and qualitative methods.
2. A) Types of surveys include phone interviews and in-person questionnaires, but I think the most useful in this day and age is computerized surveys. By promoting the survey on social media platforms or via email, you can reach a much wider scope of your target audience.
B) There are three qualitative methods that I found important as well. These are focus groups, in-depth interviews, and anecdotal feedback mechanisms. Anecdotes are especially useful for bringing to life the people behind the statistics of your program.
Chapter 25:
1. Chapter 25 is about using different types of feedback to improve the quality of your program.
2. A) Real-time feedback allows you to get interactive and immediate results while your program is still in progress. Among other things, this will help summarize your research so that you can make improvements for the future rather than at the end when there is no point to move forward.
B) Finally, you should write an evaluation report. This will help you consider changes to the program in order to enhance it as well as offer ideas for future opportunities.
Question:
How could you evaluate a campaign at the end of the program effectively? Wouldn’t it always be better to conduct evaluations during the campaign to improve in real-time?
Ch. 22 Evaluation Basics
ReplyDelete1. The importance of evaluation is sometimes overlooked or unrealistic goals are set.
A. Evaluating important questions such as what should we do, how are we doing it, and did we do it, tells marketers if the campaign is clearly working or not.
B. Evaluation should be taking place throughout the entire process, not just done at the end of a campaign.
Ch. 23 Evaluation Design
1. Look to see if other research has been done on this topic, compare your data to past marketing campaigns for similar ideas/products etc.
A. Environmental changes, policy and regulation and behavioral outcomes can all be found by designing an appropriate evaluation.
B. The most common method used is surveys in evaluations. Measures of knowledge, attitudes, beliefs, behaviors, stage of change, and demographics can all be found from a good evaluation design
Ch. 24 Evaluation Methods
1. Evaluation gives us tangible evidence that can be used to determine the success or lack of success for a marketing campaign.
A. Although surveys are the most commonly used, the observation method measures people’s actual behavior in a given situation
B. Qualitative data may not be used as much as quantitative data
Ch. 25 Using Feedback to Improve Your Program
1. A campaign is only effective if you have observable behavior change from your target audience
A. Feedback allows you to make improvement sin the future, what aspects need more work or less attention
B. Mobile feedback is also on the rise and used to improve programs. Conversation, not content, is key.
Q: Are there other ways to evaluate social marketing campaigns on twitter other than diagnostic tests?
CH 22
ReplyDeleteGood evaluation processes are central to understanding the success of a campaign.
1. There are three key questions: What should we do? How are we doing? Did we do it?
2. If evaluation is not carried out properly, vital lessons may be missed.
CH 23
The design of an evaluation strategy determines how a strategy will be evaluated.
1. There must be a focus on the target audience.
2. Using data collected by different organisations can be useful in keeping costs down.
CH 24
There are a broad range of evaluative methods used to analyse the success of a campaign.
1. Observation, surveys and qualitative research are oft-used tools.
2. A useful concept is that of the control group, which allows you to measure the effectiveness of particular variables relating to the campaign.
CH 25
Evaluative feedback can help to cast new light on the effectiveness of a strategy so that it can be improved in the future.
1. Evaluation can be carried out during or after a campaign.
2. The former strategy allows a campaign to be amended as it is going on, ironing out creases in implementation as they arise, while the latter involves snowballing observations and feedback into the next strategy.
Q: What are some case studies in misguided evaluation strategies?
Chapter 22:
ReplyDelete1. Why social marketing programs must be evaluated
2. A) There are 3 different types of evaluation of a social marketing campaign. It all starts with the formative evaluation that answers the question “What should we do?” The second one is the process evaluation that answers the following question: “How are we doing?” The last type of evaluation answers the question “Did we do it?” That is the summative evaluation.
B) The main challenges you might face in your evaluation include: unrealistic expectations, limited resources, reliance on a single method, using the wrong model, asking the wrong questions, technical problems, resistance from program staff or participants, waiting until the program is over to start evaluation and failure to use evaluation results.
Chapter 23:
1. How to structure your evaluation design.
2. A) individual-level or community-based indicators will tell you whether you have achieved your social marketing objectives.
B) Individual –level indicators encompass measures such as the following: knowledge, attitudes, beliefs, behaviors, stage of change, demographics.
Chapter 24:
1. Different evaluation methods.
2. A) Observation is another method used to collect data. Observation can be efficient, people are observed in their natural environments without awareness. Data can be accurate with this method.
B) Surveys can be: computerized surveys, in person interviews, mailed questionnaires, or telephone interviews.
Chapter 25:
1. The use of feedback to improve your social marketing program
2. A) By responding to significant information promptly, you can still improve your program when it counts. That’s why feedback shouldn’t wait until the end of the program. Usually waiting until the end doesn’t make it possible to make changes and/or improvements because it is too late.
B) The use social media helps you achieve more practical results and real-time actionable. You can get an idea of what people think about your campaign while it is running, therefore allowing you to make amendments if necessary.
Question: How are you sure an evaluation method is the best for your kind of program? How do you choose them best?
Chapter 22
ReplyDelete1. It is important to evaluate the task at hand so you don't set unrealistic goals for yourself.
2. A- You can use a summative evaluation to gauge your progress. The first question is "What should we do" followed by "How are we doing", and you conclude with "Did we do it". This series of questions assures you are staying on track, meeting goals, and completing the tasks you agreed to complete.
B- You should evaluate progress at multiple points through your project. There are different ways to measure progress, including focus groups, check ins, etc. The evaluations should be used to make changes once you find your weaknesses.
Chapter 23
1. This chapter discusses the appropriate way to structure your evaluation design.
2. A- You will need to set a specific target audience. This will help you to understand the feedback you receive and put it in perspective.
B- You can compare your target audience from two different points. Poll them before you implement your materials, and again after. You can get a good understanding of their behavioral changes that way.
Chapter 24-
1. You can measure your success through various methods including surveys, observation, and quantitative research.
2 A- Observation is simple and hands off. You just let your subjects do as they naturally would and keep an eye on their behavior. In some instances this is more applicable than others.
B- Surveys can take on many different platforms and offer a quick and easy gauge of users feelings. Be aware that your survey is structured efficiently and does not over-exert the person taking it, ensuring people are willing to complete it.
Chapter 25-
1. Once you have gathered your feedback it is important to put it to good use.
2. A- You can gather and use feedback while your program is still going on. If you choose to do so you can make necessary improvements and give yourself chances at a better outcome.
B- Social media is good for real time feedback. You can get up close and personal with individuals and do mass surveys of awareness and success.
Is there such a thing as too much evaluation?