Socw 6301 -5 | SOCW 6301 – Social Work Practice Research I | Walden University

 

Group Research Designs for Social Work Knowledge Development and Practice Evaluation

Once you have a fairly narrow research question and have conducted a thorough literature review, the next step is to turn your attention to the research design. The research design is simply a plan for how you will conduct your research, as it informs each step of the research process. In addition to research design, you are introduced to the concepts of internal and external validity and generalizability. During the latter half of the week, you focus on a particular instrument of measurement: the pretest and posttest. Capturing information from research study participants both before and after a particular event can yield priceless data.

Learning Objectives

Students will:
  • Classify types of group research designs
  • Analyze possible study limitations
  • Analyze relative internal and external validity
  • Analyze generalizability
  • Analyze a pretest–posttest in a case study
  • Evaluate research designs to assess program effectiveness
  • Analyze potential lessons learned from attrition

Learning Resources

Note: To access this week’s required library resources, please click on the link to the Course Readings List, found in the Course Materials section of your Syllabus.

Required Readings

Yegidis, B. L., Weinbach, R. W., & Myers, L. L.  (2018). Research methods for social workers (8th ed.). New York, NY:  Pearson.
Chapter 5, “Quantitative Research” (pp. 100-125)

Plummer, S.-B., Makris, S., & Brocksen S. M. (Eds.). (2014). Sessions: Case histories. Baltimore, MD: Laureate International Universities Publishing. [Vital Source e-reader].
The Hernandez Family

 

The Hernandez Family

Juan Hernandez (27) and Elena Hernandez (25) are a married Latino couple who were referred to the New York City Administration for Children Services (ACS) for abuse allegations. They have an 8-year-old son, Juan Jr., and a 6-year-old son, Alberto. They were married 7 years ago, soon after Juan Jr. was born. Juan and Elena were both born in Puerto Rico and raised in Queens, New York. They rent a two-bedroom apartment in an apartment complex where they have lived for 7 years. Elena works as babysitter for a family that lives nearby, and Juan works at the airport in the baggage department. Overall, their physical health is good, although Elena was diagnosed with diabetes this past year and Juan has some lower back issues from loading and unloading bags. Both drink socially with friends and family. Juan goes out with friends on the weekends sometimes to “blow off steam,” having six to eight beers, and Elena drinks sparingly, only one or two drinks a month. Both deny any drug use at all. While they do not attend church regularly, both identify as being Catholic and observe all religious holidays. Juan was arrested once as a juvenile for petty theft, but that has been expunged from his file. Elena has no criminal history. They have a large support network of friends and family who live nearby, and both Elena’s and Juan’s parents live within blocks of their apartment and visit frequently. Juan and Elena both enjoy playing cards with family and friends on the weekends and taking the boys out to the park and beach near their home.

ACS was contacted by the school social worker from Juan Jr.’s school after he described a punishment his parents used when he talked back to them. He told her that his parents made him kneel for hours while holding two encyclopedias (one in each hand) and that this was a punishment used on multiple occasions. The ACS worker deemed this a credible concern and made a visit to the home. During the visit, the parents admitted to using this particular form of punishment with their children when they misbehaved. In turn, the social worker from ACS mandated the family to attend weekly family sessions and complete a parenting group at their local community mental health agency. In her report sent to the mental health agency, the ACS social worker indicated that the form of punishment used by the parents was deemed abusive and that the parents needed to learn new and appropriate parenting skills. She also suggested they receive education about child development because she believed they had unrealistic expectations of how children at their developmental stage should behave. This was a particular concern with Juan Sr., who repeatedly stated that if the boys listened, stayed quiet, and followed all of their rules they would not be punished. There was a sense from the ACS worker that Juan Sr. treated his sons, especially Juan Jr., as adults and not as children. This was exhibited, she believed, by a clear lack of patience and understanding on his part when the boys did not follow all of his directions perfectly or when they played in the home. She mandated family sessions along with the parenting classes to address these issues.

During the intake session, when I met the family for the first time, both Juan and Elena were clearly angry that they had been referred to parenting classes and family sessions. They both felt they had done nothing wrong, and they stated that they were only punishing their children as they were punished as children in Puerto Rico. They said that their parents made them hold heavy books or other objects as they kneeled and they both stressed that at times the consequences for not behaving had been much worse. Both Juan and Elena were “beaten” (their term) by their parents. Elena’s parents used a switch, and Juan’s parents used a belt. As a result, they feel they are actually quite lenient with their children, and they said they never hit them and they never would. Both stated that they love their children very much and struggle to give them a good life. They both stated that the boys are very active and don’t always follow the rules and the kneeling punishment is the only thing that works when they “don’t want to listen.”

They both admitted that they made the boys hold two large encyclopedias for up to two hours while kneeling when they did something wrong. They stated the boys are “hyperactive” and “need a lot of attention.” They said they punish Juan Jr. more often because he is particularly defiant and does not listen and also because he is older and should know better. They see him as a role model for his younger brother and feel he should take that responsibility to heart. His misbehavior indicates to them that he is not taking that duty seriously and therefore he should be punished, both to learn his lesson and to show his younger brother what could happen if he does not behave.

During the intake meeting, Juan Sr. stated several times that he puts in overtime any time he can because money is “tight.” He expressed great concern about having to attend the parenting classes and family sessions, as it would interfere with that overtime. Elena appeared anxious during the initial meeting and repeatedly asked if they were going to lose the boys. I told her I could not assure her that they would not, but I could assist her and her husband through this process by making sure we had a plan that satisfied the ACS worker’s requirements. I told them it would be up to them to complete those plans successfully. I offered my support through this process and conveyed empathy around their response to the situation.

The Hernandez Family

Juan Hernandez: father, 27

Elena Hernandez: mother, 25

Juan Hernandez Jr.: son, 8

Alberto Hernandez: son, 6

Together we discussed the plan for treatment, following the requirements of ACS; they would attend a 12-week Positive Parenting Program (PPP) along with weekly family sessions. In an effort to reduce some of the financial burden of attending multiple meetings at the agency, I offered to meet with the family either just before or immediately after the PPP so that they did not have to come to the agency more than once a week. They agreed that this would be helpful because they did not have money for multiple trips to the agency, although Juan Sr. stated that this would still affect his ability to work overtime on that day. I asked if they had any goals they wanted to work toward during our sessions. Initially they were reluctant to share anything, and then Elena suggested that a discussion on money management would be helpful. I told them I would be their primary contact at the agency—meeting with them for the family sessions and co-facilitating the PPP group with an intern. I explained my limitations around confidentiality, and they signed a form acknowledging that I was required to share information about our sessions with the ACS worker. I informed them that the PPP is an evidenced-based program and explained its meaning. I informed them that there is a pre- and post-test administered along with the program and specific guidelines about missed classes. They were informed that if they missed more than three classes, their participation would be deemed incomplete and they would not get their PPP certification.

Initially, when the couple attended parenting sessions and family sessions, Juan Sr. expressed feelings of anger and resentment for being mandated to attend services at the agency. Several times he either refused to participate by remaining quiet or spoke to the social worker and intern in a demeaning manner. He did this by questioning our ability to teach the PPP and the effectiveness of the program itself, wanting to know how this was going to make him a better parent. He also reiterated his belief that his form of discipline worked and that it was exactly what his family members used for years on him and his relatives. He asked, “If it worked for them, why can’t that form of punishment work for me and my children?” He emphasized that these were his children. He maintained throughout the sessions that he never hit his children and never would. Both he and Elena often talked about their love for their children and the devastation they would feel if they were ever taken away from them.

Treatment consisted of weekly parenting classes with the goal of teaching them effective and safe discipline skills (such as setting limits through the use of time-out and taking away privileges). Further, the classes emphasized the importance of recognizing age-appropriate behavior. We spent sessions reviewing child development techniques to help boost their children’s self-esteem and sense of confidence. We also talked about managing one’s frustration (such as when to take a break when angry) and helping their children to do the same.

Family sessions were built around helping the family members express themselves in a safe environment. The parents and the children were asked to talk about how they felt about each other and the reason they were mandated to treatment. They were asked to share how they felt while at home interacting with one another. I thought it was of particular importance to have them talk about their feelings related to the call to ACS, as I was unsure how Juan Sr. felt about Juan Jr.’s report to the social worker. It was necessary to assist them with processing this situation so that there were no residual negative feelings between father and son. I asked them to role-play—having each member act like another member of the household. This was very effective in helping Juan Sr. see how his boys view him and his behavior toward them when he comes home from work. As a result of this exercise, he verbalized his newfound clarity around how the boys have been seeing him as a very angry and negative father.

I also used sessions to explore the parents’ backgrounds. Using a genogram, we identified patterns among their family members that have continued through generations. These patterns included the use of discipline to maintain order in the home and the potentially unrealistic expectations the elders had for their children and grandchildren. Elena stated that she was treated like an adult and had the responsibilities of a person much older than herself while she was still very young. Juan Sr. said he felt responsible for bringing money into the home at an early age. He was forced by his parents to get working papers as soon as he turned 14. His paychecks were then taken by his parents each week and used to pay for groceries and other bills. He expressed anger at his parents for encouraging him to drop out of high school so that he could get more than one job to help out with the finances.

Other sessions focused on the burden they felt related to their finances and how that burden might be felt by the boys, just as Juan Sr. might have felt growing up. In one session, Juan Jr. expressed his fears of being evicted and the lights being turned off, because his father often talked of not having money for bills. Both boys expressed sadness over the amount of time their father spent at work and stressed their desire to do more things with him at night and on the weekends. Both parents stated they did not realize the boys understood their anxieties around paying bills and felt sad that they worried about these issues. We also took a couple of sessions to address money management. We worked together to create a budget and identify unnecessary expenses that might be eliminated.

Key to Acronyms

ACS: Administration for Children Services

PPP: Positive Parenting Program

It was clear that this was a family that loved each other very much. Juan Sr. and Elena were often affectionate with each other and their sons. Once the initial anger subsided, both Juan Sr. and Elena fully engaged in both the family sessions and the PPP. We assessed their progress monthly and highlighted that progress. I also was aware that it was important to learn about the Hernandez family history and culture in order to understand their perspective and emotions around the ACS referral. I asked them many questions about their beliefs, customs, and culture to learn about how they view parenthood, marriage roles, and children’s behaviors. They were always open to these questions and seemed pleased that I asked about these things rather than assumed I knew the answers.

During the course of treatment they missed a total of four PPP classes. I received a call from Elena each time letting me know that Juan Sr. had to work overtime and they would miss the class. She was always apologetic and would tell me she would like to know what they missed in the class so that she could review it on her own. During a call after the fourth missed parenting class, I reminded Elena that in order to obtain the certificate of completion, they were expected to attend a minimum of nine classes. By missing this last class, I explained, they were not going to get the certificate. Elena expressed fear about this and asked if there was any way they could still receive it. She explained that they only had one car and that she had to miss the classes when Juan Sr. could not go because she had no way of getting to the agency on her own. I told her that I did not have the authority to change the rules around the number of classes missed and that I understood how disappointed she was to hear they would not get the certificate. When I told her I had to call the ACS worker and let her know, Elena got very quiet and started to cry. I spoke with her for a while, and we talked about the possible repercussions.

I met with my supervisor and informed her of what had occurred. I knew I had to tell the ACS worker that they would not receive the certificate of completion this round, and I felt bad for the situation Juan Sr. and Elena and their boys were now in. I had been meeting with them for family sessions and parenting classes for almost three months by this point and had built a strong rapport. I feared that once I called the ACS worker, that rapport would be broken and they would no longer want to work with me. I saw them as loving and caring parents who were trying the best they could to provide for their family. They had been making progress, particularly Juan Sr., and I did not want their work to be in vain.

I also questioned whether the parenting and family sessions were really necessary for their situation. I felt there was a lack of cultural competence on the part of the ACS worker—she had made some rather judgmental and insensitive comments on the phone to me during the referral. I wondered if there was a rush to judgment on her part because their form of discipline was not commonly used in the United States. In my own professional opinion, some time-limited education on parenting and child development would have sufficed, as opposed to the 3-month parenting program and family sessions.

My supervisor and I also discussed the cultural competence at the agency and the fact that the class schedule may not fit a working family’s life. We discussed bringing this situation to a staff meeting to strategize and see if we had the resources to offer the PPP multiple times during the week, perhaps allowing clients to make up a class on a day other than their original class day.

I met with Elena and Juan Sr. and let them know I had to contact the ACS worker about the missed classes. I explained that this was something I had to do by law. They told me they understood, although another round of parenting classes would be a financial burden and they had already struggled to attend the current round of classes each week. I validated their concerns and told them we were going to look at offering the program more than once a week. I also told them that when I spoke to the ACS worker, I would also highlight their progress in family and parenting sessions.

I called the ACS worker and told her all the positive progress the parents had made over the previous 3 months before letting her know that they had missed too many classes to obtain the PPP certificate. The ACS worker was pleased with the progress I described but said she would recommend to her supervisor that the parents take the PPP over again until a certificate was obtained. She would wait to hear what her supervisor’s decision was on this matter. She said that family sessions could end at this point. In the end, the supervisor decided the parents needed to come back to the agency and just make up the four classes they missed. Elena and Juan Sr. were able to complete this requirement and received their certificate, and the ACS case was closed. They later returned on their own for a financial literacy class newly offered at the agency free of charge.

Choose One of the Following Articles:

Bauman, S. (2006). Using comparison groups in school counseling research: A primer. Professional School Counseling, 9(5), 357–366.
Retrieved from Walden Library databases.

Kohl, P. L., Kagotho, J., & Dixon, D. (2011). Parenting practices among depressed mothers in the child welfare system. Social Work Research, 35(4), 215–225.
Retrieved from Walden Library databases.

Leung, P., Erich, S., & Kanenberg, H. (2005). A comparison of family functioning in gay/lesbian, heterosexual and special needs adoptions. Children and Youth Services Review, 27(9), 1031–1044.
Retrieved from Walden Library databases.

Moran, J., & Bussey, M. (2007). Results of an alcohol prevention program with urban American Indian youth. Child & Adolescent Social Work Journal, 24(1), 1–21.
Retrieved from Walden Library databases.

Pinderhughes, E. E., Dodge, K. A., Bates, J. E., Pettit, G. S., & Zelli, A. (2000). Discipline responses: Influences of parents’ socioeconomic status, ethnicity, beliefs about parenting, stress, and cognitive-emotional processes. Journal of Family Psychology, 14(3), 380–400.
Retrieved from Walden Library databases.

Southerland, D., Mustillo, S., Farmer, E., Stambaugh, L., & Murray, M. (2009). What’s the relationship got to do with it? Understanding the therapeutic relationship in therapeutic foster care. Child & Adolescent Social Work Journal, 26(1), 49–63.
Retrieved from Walden Library databases.

 

Group Research Designs

There are several different types of research designs. Each design is intended to respond to a particular type of research question. The type of research design depends on the type of research questions asked. For this Discussion, select one of the articles from the reading list and consider several classifications of group research designs.

By Day 3

Post your response to the following: Describe which groups are compared in the research. Then, classify the research design as follows:

  1. By explaining whether the study is pre-experimental (cross-sectional, one-shot case study, and longitudinal), experimental (control group with pretest and posttest, posttest only, or four-group design), or quasi-experimental (comparing one group to itself at different times or comparing two different groups)
  2. By indicating what the researchers report about limitations of the study
  3. By explaining concerns you have regarding internal validity and the ability of the study to draw conclusions about causality
  4. By explaining any concerns you have about the generalizability of the study (external validity) and what aspect of the research design might limit generalizability

Please use the resources to support your answer.

By Day 5

Respond to a colleague’s post by explaining an insight you gained from this week’s Resources that addresses a topic in your colleague’s post. Please use the resources to support your post.

  

Discussion 2: Pretests/Posttests and Attrition

This week’s video introduces you to the Hernandez family. Juan and Elena Hernandez are mandated to attend parenting classes. As part of the parenting classes, they are required to participate in both a pretest (taken before classes begin) and a posttest (taken after classes end). Pretests and posttests can yield very useful information. They can measure the learning that has or has not taken place as a result of the education being provided. For this Discussion, imagine what kinds of questions would be on a pretest and posttest for this particular study sample (the parents who are taking parenting classes).

By Day 5

Post your explanation of what you think the social workers wanted to know when they designed the pre/posttest for evaluating the parenting classes. Be sure to address the concept of attrition and whether or not you believe they took that into account. Be sure to explain why. Then, analyze potential lessons learned from attrition. Finally, explain how you might design a study to gather data about the effectiveness of these parenting classes. Use the following questions to focus your thinking:

  • Would you recommend a pre-experimental, experimental, or quasi-experimental design?
  • Would the study involve measurement over time?
  • Would there be a control group?
  • In this group research design you imagine, what or who will be compared?
  • What limitations in terms of generalizability and internal validity can you anticipate based on the research plan you envision?
  • What can you tell the social worker about the issue of client drop out (also called attrition or experimental mortality)?
  • If there is no control group, what type of research design could answer the question: Is this parent training class effective in reducing abusive parenting practices?

Please use the resources to support your answer.

By Day 7

Respond to a colleague’s post by supporting or refuting ideas about his or her vision of how the study might be designed to gather data about the effectiveness of these parenting classes. Please use the resources to support your answer.

Melissa 

 RE: Discussion 2 – Week 5COLLAPSE

The pre-test/post-test was another research method that contributed to the validity of the research.  Pre-test showed where the participants stood at the beginning of the research, and the post-test measured the progress that was made by the end of the research.  I do not believe that attrition was considered initially.  The social workers in the video were too confused about their next set of actions after they were forced to drop Mr. Hernandez (Laureate Education, 2013).  Attrition, only then, became a factor.  Attrition is a strong potential in a research study and must be considered in advance. 

Research Method

The research method that I would recommend is a quasi-experimental design.  Quasi-experimental design is an empirical interventional study used to estimate the causal impact of an intervention on a target population without random assignment (Yegidis, Weinbach & Myers, 2018).  Funding depends on the validity of the research study.  The more effective this program is to individuals and families, the more funding the program will receive.  Due to the topic of the group and the reason for the research study, internal validity does not need to be manipulated.  Attrition affects the effectiveness of the study which is the determining factor of funding.

Measurement over Time

The study should involve measurement over time.  Measurement over time offers stability and validity to the research, making the conclusion concrete.  There would be a control group.  A control group is compared to the experimental group but does not receive the test variable.  This research method is used to find answers within the experiment.  This method accounts for the reliability of the research as well. 

Research Design

In this group research design that I imagine, two groups will be measured.  This research method will measure the control group which will be the individuals that have not met the attendance requirements and then the experimental group, which will be the individuals that have successfully completed the program.  This research method will measure the effectiveness of the program in terms of reduction of undesired behaviors and attrition for the group that was partially observed. 

Limitations

Limitations in terms of generalizability and internal validity that I anticipate based on the research plan that I envision include many individuals that will not complete the program or not be available for post-testing.  This alters the validity of the study as well as the conclusion.

Attrition

Pertaining to the client drop out, I can tell the social worker that attrition is inevitable and should be expected and accounted for early in the research process.  Individuals not completing the research study or not being available for a post-test is an often occurrence and should be expected by the researcher.  Adding attrition into the results is beneficial to the study by helping to maintain stability.

No Control Group

If there is no control group, the research design that could answer the question, “is this parent training class effective in reducing abusive parenting practices,” is the pre-experimental design.  The pre-experimental design studies a single group meaning no control group nor comparison involved in this research design (Yegidis, Weinbach & Myers, 2018).

References

Laureate Education (Producer). (2013). Hernandez family (Episode 5) [Video file]. In Sessions. Retrieved from https://class.waldenu.edu.

Yegidis, B. L., Weinbach, R. W., & Myers, L. L. (2018). Research methods for social workers (8th ed.). New York, NY: Pearson. Chapter 5, “Quantitative Research” (pp. 1Laquita Renwrick RE: Discussion 2 – Week 5

Pretest and posttest are fundamental methods of inquiring and measuring data obtained from client systems about the effectiveness of a program or service. Social workers are ethically bond to improving and increasing accessibility of services to address the needs of clients.  When social workers were designing pre and post test for parenting classes, they wanted to assess what the parents knew before starting classes and to evaluate the effectiveness of the class. The evaluation of the effectiveness can be determined by the parents ability to demonstrate skills and comprehend the educational portion of the class, all of which could be identified with the completion of the posttest.  

This writer would recommend a pre-experimental design due to its ability to measure the change experienced by parents after receiving intervention (classes). Pre-experimental  designs do not have control groups to utilize for observational purposes. Pre-experimental designs do not attempt to control threats to internal validity of research (Yegidis, Weinbach, & Myers, 2018). The benefit of selecting this design is that researchers only compare the individuals to themselves, observing improvement of behaviors and skills due to the implication of interventions because it is assumed that one’s ability to change can only be attributed to treatment.

The study would involve measurement over time. I think research is better studied when it’s longitudinal so that data can be measured with the exposure of a variable. There should be no control group because comparing parenting skills of one group to another would not truly represent improvement as it is based on individuality. The goal of the study would be to determine if the program was culturally fitting and exemplified relativity to the population or sample being served. Limitations that can be anticipated based on interval validity are self-reporting of data and cultural bias. It has to be anticipated that study participants may not b truthful and without visual supervision there is no way to determine if the parent have modified parenting skills. The other limitation is cultural bias of the study.  Cross cultural research is the study of human behavior, family systems, and other social organizations in cultural contexts of the selected sample. The study has to culturally competent and take in account diversity of the selected sample population. The issue of experimental mortality can be attributed to social factors that hinder or prohibit client from maintaining attendance. As a social worker, addressing client barriers and providing assistance can encourage compliance with treatment. Therefore, using per-experimental designs would be effective in determining if parenting classes are  effective in reducing abusive parenting practices.

 

Assignment: Locating an Empirical Research Article

Empirical research articles document a study that is either quantitative, qualitative or a mixed methods research design. When authors write an empirical research article they typically follow a format that looks like this: Introduction/Background, Literature Review, Methodology, Findings, and Discussion. The authors recount literature on their specific research topic and describe in a systematic manner how the data was collected and then analyzed in order to answer the research question(s). Once the data is analyzed, they present the findings. Finally, they interpret the findings using past literature to help understand the findings.

What we broadly describe as a “quantitative study” includes numerical summaries that involve descriptive statistics (averages, standard deviations), correlations, and inferential statistics (such as T-tests, Chi Squares and other kinds of analyses). These kinds of studies can include certain elements such as per- and post-tests or survey results looking at correlations between variables.

Qualitative articles, on the other hand, use interviews, focus groups, observations, and written answers to questions. Rather than using statistics to summarize the study, these studies look at themes and present the material using words, phrases and often paragraphs to illustrate what they are representing.

To prepare for this assignment, review Week 1’s readings and resources on how to locate an empirical research article using the library’s databases.

For this Assignment,

  • Locate an empirical research article that is either a quantitative or qualitative study from a peer reviewed social work journal for the final assignment.
    • Do not select an empirical research article that describes a mixed methods study. The reason is because a mixed method study involves both a quantitative and qualitative component. You would have to do two reviews – one for the quantitative component and one for the qualitative component — for the final assignment.
  • Upload the article. Your instructor will review the article to make sure it is an empirical research article and will approve it for your use for the final assignment.

 Angelica Wiggins RE: Discussion 1 –

Parenting Practices among Depressed Mothers in the Child Welfare System

The purpose of this study was to analyze a nationally representative sample of families referred to Child Protective Services (CPS) agencies, the National Survey of Child and Adolescent Well-Being, to examine the association between maternal depression and parenting practices over a 36-month follow-up period after a CPS investigation was completed (Kohl, Kagotho & Dixon, 2011). The interaction between depression and time was also analyzed for each parenting practice to determine how changes in maternal depression affected changes in parenting. This study is a quasi-experimental study as it is comparing parenting practices between depressed and nondepressed mothers. Maternal depression is the control group. The dependent variables were three parenting practices: harsh parenting, neglect, and emotional maltreatment. 

The limitations reported in this study is that the sample was only limited to combinations in which the child remained in the home after a maltreatment investigation by CPS. There were no measurements or reports of depression being measured if the child was removed from the home. That represents gaps in the study.

Concerns that I have regarding internal validity and the ability of the study to draw conclusions about causality are based on the fact that self-reported emotional maltreatment remained high across time points. With the gaps present in the study, how valid are these reports? Does it seem biased?

Concerns in regard to the generalizability of the study and what aspect of the research design might limit generalizability include implications of this work are the needs for better identification of mental health needs for mothers entering the child welfare system and parent training to specifically address positive parenting.

Reference

Kohl, P. L., Kagotho, J., & Dixon, D. (2011). Parenting practices among depressed mothers in the child welfare system. Social Work Research, 35(4), 215–225. RetrieDiane Sharkey RE: 

  1. Explain whether the study is pre-experimental (cross-sectional, one-shot case study, and longitudinal), experimental (control group with pretest and posttest, posttest only, or four-group design), or quasi-experimental (comparing one group to itself at different times or comparing two different groups).
     

The study I decided to use was classic experimental design, or pretest–posttest control group design (Yegidis, Weinbach, Myers, 2018, p. 113). The design included random sampling, experimental group with pretest and post-test, and a control group with both pre and post-tests.

*The experiment did include follow-up post-tests at six and twelve months, but I couldn’t find if this classified the design as something different.

  1. Researchers report about limitations of the study.
     

First, the researchers report transiency of members in both groups as a limitation because they are unable to obtain post-test data from approximately 25 percent of the participants who took the pretest (Bauman, 2006, p. 363). Thirty-five percent of the data was missing at the six-month follow-up and almost half (43%) at the 12-month follow up (Bauman, 2006, p. 363).

Second, school climate was not conducive to support effective implementation of the new strategies (Bauman, 2006). Therefore, impacting long-term desired results.

Third, students in the experimental group could have discussed the program with students in control group (Bauman, 2006). Researchers also suggest students in the control group could have learned some of the methods by observing the behaviors of participants in the experimental group (Bauman, 2006).

  1. Concerns regarding internal validity and the ability of the study to draw conclusions about causality.
     

One concern I have regarding internal validity is experimental mortality from the transiency of members in both groups (Bauman, 2006). By the six month follow-up nearly half of the participants were no longer available for data collection, therefore making it difficult to infer long-term impact and causality (Bauman, 2006).

A second concern I have about internal validity if from a limitation, experimental group participants modeling desired behaviors for and/or discussing the program with control group participants (Bauman, 2006). This could have a desired or undesired effect on the control group. Control group participants could see the interventions implemented, the positive consequences from teachers and administration and want to emulate the strategies to achieve the same positive responses. However, the control group participants could also go the opposite direction and be angered by the extra attention given to the experimental group members, causing them to lash out which could have deleterious consequences.

  1. Concerns about the generalizability of the study (external validity) and what aspect of the research design might limit generalizability.
     

As a middle school teacher, I know that in each year of middle school (sixth, seventh, and eighth grades) children change drastically. This is due, in large part, to biological changes which impact social and psychological aspects of adolescent’s lives. Therefore, testing all grades in the middle schools would potentially impact the internal validity of the experiment regarding maturation (Yegidis et al., 2018). However, categorizing the results of both groups’ pre and post-tests separating by age group or grade level would allow researchers to identify any issues with causality. Furthermore, the study doesn’t state whether they chose three middle schools out of a total number in the city or if there were only three in that city (Bauman, 2006). This could impact generalizability if the schools were chosen based on race/majority population given the study affirms the participants were comprised of mostly African Americans (Bauman, 2006). Finally, the program was directed by African American men who were experts in the course material and were not their regular teachers (Bauman, 2006). Participant engagement and response could be different depending upon bias or preference the participants may have.

Side note-as a teacher, I also know that kids typically pay more attention to special guests especially if it’s someone they can tell is investing in their well-being.

References

Bauman, S. (2006). Using comparison groups in school counseling research: A primer. Professional School Counseling, 9(5), 357–366.

Yegidis, B. L., Weinbach, R. W., & Myers, L. L.  (2018). Research methods for social workers (8th ed.). New York, NY:  Pearson.