Formative interventions and design research
Discussions of method don’t often sound all that sexy, but I love them anyway. My first two sessions of the ICLS 2008 conference have been about method (maybe process is a better term). First, Yrjo Engestrom talked about formative interventions, an activity theory-style approach to research, and then Ilya Zitter described her process for using Educational Design Research in her doctoral work.
Not surprisingly, Engestrom railed against the “gold standard” of randomized controlled trials as the best and only way to properly conduct research. He mixed in a couple jabs at the U.S. – one for emphasizing such studies and one for making unpopular interventions. I’m with him on both. Randomized controlled trials (RCT) shouldn’t be the gold standard for all kinds of research, and the U.S. shouldn’t have intervened in Iraq. At least not the way we did. But, I digress. I was talking about method.
In contrast to the positivist RCT program, Engestrom recommends a different process entirely. His process, we’ll call it formative interventions (that was on his slides), engages the research site as a participant in the project rather than as a passive recipient of a designed intervention. It differs from ICT (and even from Design Research – an approach gaining popularity in education research) in three main ways:
- starting point,
- process, and
- outcome.
The starting point for formative interventions are poorly understood objects. RCT and design research start with some goal in mind. Having a goal presupposes that the goal is desirable. I dislike the arrogance behind starting a project from, “I know how it should be,” and so it’s no surprise that I like formative interventions’ starting point.
Engestrom calls formative interventions’ process “double stimulation.” That term doesn’t really work for me. I think what he means is that the research introduces and recognizes changes in the research environment over time. Whether those changes are planned by the researchers or not is not terribly important. The process of studying a changing phenomenon differs dramatically from the “execute, refine, repeat” approach RCT takes.
Lastly, the outcomes of the two methodological approaches differs. For Engestrom, the outcome should be “new activity concepts” and for RCT, it’s “solutions.” I’m often frustrated by “solution” terminology – because I’m uncomfortable labeling social phenomena as broken, because I’ve seen too many “solutions” that don’t have clear “problems”, because I just don’t see the world that black and white.
So now we have an outline of Engestrom’s preferred methodological approach. I like it. It’s engaged, rigorous, and embraces the ongoing and changing nature of social situations. Trouble is, it’s hard to sell, in the U.S. especially, and even harder to do.
Enter Ilya Zitter. Ilya is a PhD student at Utrecht University, and she uses a method she calls “Educational Design Research” in her doctoral work. Basically, she uses research, design, and practice approaches to study undergrads in a projects course. Hooray for higher education at ICLS! It’s almost as satisfying for me to engage as adults’ informal and workplace learning. Anyway, Ilya gave a short talk in a firehose session where she described how she conducted her research. This is exactly the kind of talk I like to attend at conferences. I can read papers, but papers about how the research was conducted are hard to come by. Sure, papers include methods sections, but those don’t often tell you the nitty gritty details. Ilya talked about her struggle to balance research, design, and practice in her work. This is a struggle I get to avoid in my dissertation but which is central to my life at Microsoft Research.
At MSR, we’re engaged in a formative intervention study of sorts. We’re working with HR and managers to adjust social and technological tools used in onboarding at Microsoft. I’m often uncomfortable in the “design” and “intervene” portions of such studies. I much prefer to be a fly on the wall. That’s not immediately useful (or publishable) though. I, like Ilya, am struggling to find balance and to negotiate relationships among researchers and practitioners all while gathering and analyzing data. It’s hard, but at least I’m not alone.
Stovepipes and how they confuse and frustrate
The title of this post suggests a treatise more than a lowly blog post, but, I'm a busy proposal writer, so I'll have to settle for the post. A friend sent me a link to a Gartner newsletter yesterday, and the title was "It's official. Collaboration is a top business priority." That's good news for me since I'm a collaboration researcher. It's nice to know collaboration is a priority for someone besides me. I clicked through the newsletter and even visited the website for a conference Gartner is hosting called the Portals, Content and Collaboration Summit. Here are some thoughts, in no particular order.
I use "stovepipes" to refer to the isolated, vertical towers of thought and work into which we've organized ourselves. I think I picked it up while serving on SI's dean search committee where we spent a lot of time talking about how we wanted to avoid stovepipes in SI. We're an interdisciplinary school, and we wanted to recommend a dean who would encourage intellectual respect and collaboration among the disciplines rather than stovepiping them. Ok, so, now I'm concerned about the stovepiping that happens between academia and the business world. This is not a new concern for me, but it's one I haven't written down for public consumption yet. It worries me that a summit like Gartner's can take place and that academia neither talks about it nor attends. I don't mean to imply that Gartner's summit is the be-all, end-all and that it's going to change the world forever, but it could be something interesting. It concerns me that when I wanted to study newcomers in an organization, I was thwarted by lack of connections to organizations. Instead, I ended up studying newcomers to an academic environment and then interviewing (a year later) for a serendipitous opportunity to study newcomers in a public company. Man, I hope I get that internship. I digress.
The divide between academic research and business research seems to be growing. Is it growing? Am I just more attuned to it now than I have been? What can I do to get more connected to businesses so that I can learn from them, and so that my work can have some real world impact? I don't want to be a messiah with academic knowledge; I want to get my hands dirty outside the lab. What would the cost to my academic reputation be if I were to focus on collaborations with industry researchers and managers? Where is the tunnel between the stovepipes?
Stovepipes aside, Gartner's summit has one of the most male-dominated presenter lists I've seen in a quite a while. I hope overall that their company is more diverse. That said, I'm curious about the summit. I do wonder what collaboration technologies are on the horizon. I'll be paying attention, for sure.
iConference Roundtable
Sean Munson and I will be hosting a roundtable discussion at the iConference at UCLA in February. The preconference wiki is up and ready for your contributions. Here's the description of the roundtable:
---------------------
Professional students, whether undergraduates or masters’ students, represent a significant portion of the iSchool community. How do iSchools effectively educate those students while continuing to develop successful research programs? This roundtable discussion will focus on how iSchools educate their professional students and engage them in the research aspect of their programs. Innovative approaches to training and integration will be the central theme of this discussion. In an iSchool – where students training for professions including librarianship, information policy, human-centered computing, preservation and researchers exploring such topics as incentive-centered design, forensic informatics, computational linguistics, and digital libraries have both competing and complimentary goals – the potentials for collaboration, innovation, misunderstanding, and disharmony are all high.
The annual iConference provides a unique opportunity for us, as a community, to discuss the roles our professional students have in shaping our identity and our practices. The proposed roundtable will invite participants to discuss questions such as:
* What should the role of research in training information professionals be?
* How can we best engage professional students in our research?
* How do iSchools address the unique curricular challenges we face in preparing students for a very wide variety of careers?
* What do we want an Information degree to signal in the marketplace?
* What are some successes in which research and professional training have benefited one another?
Participants will share innovative approaches to professional education, best practices in engaging professional students in research programs, and remaining challenges. We intend roundtable participation to represent the diversity of iSchools’ current programs. Confirmed participants include:
* Dr. Eileen G. Abels, Master's Program Director, Associate Professor, College of Information Science & Technology, Drexel University
* Dr. Judith S. Olson, Richard W. Pew Collegiate Professor of Human Computer Interaction and Associate Dean for Academic Affairs, School of Information, University of Michigan
Each speaker will present introductory remarks highlighting some of the achievements and challenges they face in their home programs, after which discussion will include questions and input from the attendees. This will be an interactive forum proposing ideas for new approaches to education and integration of professional students. We encourage participants to discuss ideas that work (and those that don’t!) in their schools. We will create and publicize a wiki space for pre- and post-conference participation as well.
---------------------
We're hoping that Beth Mynatt from Georgia Tech will also join us and talk a bit about their successes. We'd love to have anyone interested in professional students and information research to join our discussion both online and in Los Angeles. The preliminary schedule of the conference indicates that our roundtable will take place sometime on Friday, February 29.
Argh. Again.
I’m having trouble convincing myself that staying involved in big science projects is a tenable arrangement for me. Big science drives a lot of SI’s money, and I’m starting to feel a bit like a puppet. Sure, I think big science is interesting and valuable. Who doesn’t want an anthrax vaccine or concrete that bends? I just don’t want to spend all my intellectual time and energy watching people make those vaccines or bend that concrete. I’m tempted, again, to leave big science collaboration studies to someone who cares more about technology. After spending a couple days at an NSF symposium ostensibly about cyber-enabled discovery and innovation, I am even more convinced that NSF and its CISE program are not the place for me to make my splash. Sure, NSF money is nice in that it’s often big and makes work possible. But do I want to do that work? I don’t think so. This symposium has served very effectively to convince me, at least for now, that my summer enthusiasm about using these big science collaborations as cases for a general study of collaboration was foolish. Right now, there are equations being projected. Equations. I came to Troy, NY to wave the sociotechnical banner and learn about physics, apparently. I think I’d rather dump the banner and put my “social” t-shirt back on.
So now I'm done presenting, and I feel a little better. Perhaps some of my earlier crankiness was due to my stress over having not finished my presentation to include said crankiness. In the end, I got to wave the sociotechnical banner and ask for funding to support social science enabled by computation. Not bad for a day's work.
Before I hit "Publish," let me mention one more battle raging at this symposium. Gender. I was the second woman to present in as many days. We've seen a new presentation about every 20 minutes. This is not only shocking and accurate, it's unacceptable.
Sociotechnical Road Show
I'm off to Troy, NY tonight to wave the sociotechnical banner at an NSF workshop. I'll be giving a short (~10 minutes) talk on Thursday afternoon, and I'm working on my talk notes. Some of you already know I like to give a different kind of talk - minimalist slides, helpful pictures and videos, lots of movement - but I'm not sure how such a talk with fly at this workshop. I chickened out in 2005 when I gave a talk about RideNow at the GROUP Conference, but I'm going for gold this week. The role of graduate students in these workshops is unclear to me, but it's obvious that I have an audience I wouldn't normally encounter.
Here's the white paper I submitted. One of the organizers must have liked it because they invited me to come and talk during the doctoral student forum. I'll be talking about how the data generated by wireless sensing technology may be used/studied for understanding the structural health of our civil infrastructure (think bridges). Real-time data about the health of a structure could be immensely useful for engineers monitoring its needs for repair, for rescue workers responding to a fire or other calamity, and researchers looking for ways to improve structures (and wireless sensing, for that matter). It's pretty easy for me to get excited about studying how first responders and rescue workers would use such data and the information flows it produces, but I think I need to stay closer to engineers in this talk. We'll see though, I guess.
I admit, I'm nervous about the workshop. Every list of invitees or participants that I've seen is incredibly male- and computer science-dominated. I'd rather not deal with gender and disciplinary politics at every turn, but such is life. I'll try not to let the various layers of politics derail me this week. I've been given an interesting opportunity in being invited, but I'm not quite sure what that opportunity offers. I'll follow up from Troy later this week.
Back from Connections 2007
I made some notes during the Connections 2007 conference about things I should post to my blog, and I'm just getting around to doing so. The conference was very useful - especially in forcing me to prepare a coherent presentation - and I recommend it to other iSchool doctoral students. Some things I noticed:
Grad students are good speakers
Plenty of people complain about the inability of most academics to give interesting talks or appropriately use PowerPoint or similar tools. The students who presented at Connections were a couple standard deviations above the mean. Their slides had the right amount of text; their diagrams were illustrative. They seemed knowledgeable and prepared and did a thorough and thoughtful job answering questions. I hope we can all keep those presentation skills as we continue into "real" jobs. Even presentations about work far from my own had me paying attention. It was a lovely, welcome change from other conferences I've attended.
Presentation room setup is a pain
I like to move around while I present, and that was just not an option this time. Please, when designing rooms for presentations, use mobile mics (or proper acoustics that don't require mics to reach the back row), put the podium on wheels so it can be turned to accommodate remote controls, and don't require that the speaker be in the dark in order for slides to be visible. I have many other pet peeves about presentation room setup, but these three were the most troublesome during this particular presentation. I'm a Mac owner and Keynote user. I like to use one word per slide. It's important for me to be able to move around and still use my remote. I'm also pretty loud. So a room where people in the back row can't hear me has sound issues.
Keynote rocks
I love Keynote. It's presentation view and rehearse options dominate the competition. It's easy to make slides and to insert pictures and other files. I'm officially switching.
Bendable concrete is not my passion
More on this later. I presented some discussions of diffusion of innovations and identity based on what I'm seeing in the bendable concrete project. While I find those theoretical frameworks interesting, I'm not sure how sustainable my interest in the bendable concrete project really is. I understand that one must wax and wane during a dissertation, but it seems early in the game for me to be waning every time I hear about something more exciting. If you have recommendations for more personally exciting places for me to study innovation, collaboration, communities of practice, and/or identity, please share them. I'm still mad at Bill Frist for making my poker studies impossible. Stupid online gaming finance laws.
Lynette Kvasny – Forms of Capital, IT, and Economic Development
presentation sponsored by Dean's Office, CREW, CIC, and UM@SI (12-1pm Ehrlicher, 3 April 2006)
thinking about cultural and social capital rather than simply economic
What do people bring with them when they engage IT, and how does that shape their interaction?

