Social Settings and Randomized Cluster Trials: Journey of the William T. Grant Foundation Edward Seidman Event: National and International Perspectives on Place-Randomized Trials in Education Location: Institute for Human Development and Social Change, New York University Date: Friday, October 3rd, 2008 William T. Grant Foundation www.wtgrantfoundation.org Overview of the Presentation • Shifting Paradigms • WTGF’ RFPs and Research Interests in the 21st Century • Social Settings Framework • Evolving Research Design and Analysis Challenges • Evolving Setting-Level Measurement Challenges • Future Challenges and Directions William T. Grant Foundation www.wtgrantfoundation.org 2 Shifting Paradigms • Individualism (or person-centered) foci • Preoccupation of western style thought • Well developed instrumentation • Population-centered prevention/promotion • Policy perspective • Context • Seemingly all inclusive • Hard to get a handle on for: • Research, including measurement • Intervention • The proximal Social Settings of daily experience • A “place to stand” William T. Grant Foundation www.wtgrantfoundation.org Chronology of RFPs Re. Social Setting Research and Lessons Learned • 2002 RFP: Intervention Research • Naïve conceptions of intervention, research design and analysis, limited understanding of power when “program” was the unit of analysis > need for capacity-building • Need for further specification of our interests in contexts/settings •2005-7 RFP: Intervention Research to Improve YouthServing Organizations •Need for the development and improvement of the measurement of social setting processes • 2007-? RFP: Development and Improvement of the Measurement of Classroom Quality (in collaboration with the Spencer Foundation) William T. Grant Foundation www.wtgrantfoundation.org Why Social Settings? • Need to understand the proximal settings of daily experience – families, peers, classrooms and schools, youth-serving organizations, neighborhoods • Need to understand how policies work through proximal settings of daily experience to effect youth • Social settings as loci of intervention > more “bang for the buck,” especially in classrooms and schools and other youth-serving organizations in terms of prevention and promotion William T. Grant Foundation www.wtgrantfoundation.org Social Setting Theory and Measurement References • WTG Annual Report Essays: • 2005 Social Setting Theory and Measurement (website) • 2006 Why try to understand and improve social settings? (website) • 2006 Doing social setting research (website) • 2007 Measuring social settings (website) William T. Grant Foundation www.wtgrantfoundation.org Social Setting Theory and Measurement References (continued) Tseng, V., & Seidman, E. (2007). A systems framework for understanding social settings. American Journal of Community Psychology, 39(3-4), 217-228 Cohen, D. K., Raudenbush, S. W., & Ball, D. L. (2003). Resources, instruction, and research. Educational Evaluation and Policy Analysis, 25, 119-142. Pianta, R.M. (in press). Classroom management and relationships between children and teachers: Implications for research and practice. In Good, T. & Evertson, (Eds.), Handbook of Classroom Management. Shinn, M., & Yoshikawa, H. (Eds.) (2008). Towards Positive Youth Development: Transforming Schools and Community Programs. New York: Oxford Press. William T. Grant Foundation www.wtgrantfoundation.org Macro Forces Schools (Classrooms) Macro Forces Youth Outcomes Families Neighborhoods Youth Programs O.S.T. Macro Forces Macro forces are policies, the economy, the environment, secular trends in demography, and the like. 2 Resources Macro Forces Social Social Processes Settings Organization of Resources William T. Grant Foundation www.wtgrantfoundation.org Macro Forces Youth Outcomes Setting Norms Relationships/ Practices Social Processes Participation in Activities Human Physical Resources Economic Temporal Social Organization Physical Organization Temporal Organization Economic Organization Organization of Resources Evolving Research Design & Analysis Challenges • Designing Research for Place-Randomized Trials to increase precision (Raudenbush, Bloom, and colleagues): Old issues in search of new answers • N of nested unit – asymptotes quickly • N of randomly assigned units • Baseline covariates, pretests at individualand school-level • Intra-class correlation coefficients • Matching of covariates – Products: • Optimal Design Software • Consultation Service William T. Grant Foundation www.wtgrantfoundation.org Evolving Setting Measurement Challenges •How can we effectively measure setting-level processes? • Observations, logs, artifacts, etc? • Multi-method, multi-source, multi-construct? • Norms: consensus measures? • Reliability of measuring setting-level processes = Generalizability studies, e.g., CLASS, YPQA, & others •Linking setting-level change to individual-level change William T. Grant Foundation www.wtgrantfoundation.org Future Challenges and Directions • Using new knowledge accrued to better plan and reduce costs of future place randomized trials •Disentangling dynamic nature of setting-level constructs •Discovery of new or improvement of analytic methods to establish causality between settinglevel and individual-level change William T. Grant Foundation www.wtgrantfoundation.org