Photo: MFA Boston
The results of a study on school field trips surprised the Brookings Institution researchers.
One of the messages I take from this Brookings report on student field trips is the importance of conducting research with both an open mind and a willingness to follow up on unexpected results. Another takeaway: researchers prefer to study things for which there are a lot of data available; worthwhile questions that don’t have a lot of data to work with often don’t get studied.
Jay P. Greene writes, “Most education research focuses on math and reading outcomes or educational attainment because those are the measures that the state collects and are readily available to us. Less is known about how students are doing in other subjects and whether their progress in those areas has important benefits for them and society. …
“A new experiment … examines long-term effects of students receiving multiple field trips to the Woodruff Arts Center in Atlanta. The Woodruff Arts Center houses the High Art Museum, Alliance Theater, and Atlanta Symphony, all on one campus.
“We randomly assigned 4th and 5th grade school groups to get three field trips per year – one to each of Woodruff’s arts organizations – or to a control condition in which students received a single field trip. …
“The surprising result is that students who received multiple field trips experienced significantly greater gains on their standardized test scores after the first year than did the control students. …
“The reason these results are so surprising is that previous research had suggested that arts instruction tended not to ‘transfer’ into gains in other subjects. …
“When we conducted the analysis on the effects of treatment on test scores, we expected to find no statistically significant effects, just like almost all previous rigorous research. …
“We still do not believe that arts instruction and experiences have a direct effect on math or ELA ability. We think this because the bulk of prior research tells us so, and because it is simply implausible that two extra field trips to an arts organization conveyed a significant amount of math and ELA knowledge.
“Our best guess is that test scores may have risen because the extra arts activities increased student interest and engagement in school. … Maybe arts-focused field trips do not teach math or reading, but they do make students more interested in their school that does teach math and reading. But this is just a guess. …
“The odd thing about trying to write a paper with these results to present at conferences and submit to a journal is that there is strong pressure for us to pretend like we expected our findings all along. Discussants and reviewers generally don’t want to hear that you found something you didn’t expect and don’t really know why. They want to hear a clean story about how your results make sense and follow from your theory and literature review. In short, social science favors the false appearance of confidence.”
Ah, yes. When I was at the Fed, I often wondered about such things, but as a non-economist, I knew I was out of my depth.
Read about how the researchers intend to unpack the meaning of their unexpected results with future studies, here.
Suzanne's Mom's Blog 