What’s all this talk about big data?

Maybe it hasn’t popped up on your radar yet, but it seems like everywhere one turns these days there’s another perfectly coiffed Nostradamus-impersonator lauding the inevitable big data revolution that’s just around the corner for higher education.

In case you’re wondering what I think about big data and all of the hubbub about it, I’ve shared a link to something I wrote for the Chronicle of Higher Education recently that they titled, “Big Data, Scant Evidence.” If you can’t access the it from where you are reading this post but really want to read the piece, send me a note and I’ll try to get an unlocked copy to you. My article is part of a larger supplement published last week about the big data trend in higher education. You might find some of the other articles interesting, although it’s hard to read some of this stuff and not think, “Isn’t this what we’ve been doing at Augustana for a while now?” Well . . . yes. Except that we aren’t necessarily a big enough place to produce big data. So what do we call our data? Diminutive? Pocket-sized? Lean?

Whatever you want to call it, we seem to be pretty good at improving based upon solid information.

Make it a good day,

Mark

A Shameless Plea

First or all, I owe you all a hearty heap of thanks for your patience this spring. For a couple of reasons, some of which can be chalked up to coincidence and some of which can be blamed squarely on me, we are participating in more than the usual number of surveys this spring.

Of course, I’d be remiss if I didn’t say something about the awesome data that we will have at the end of this term and how much it will likely inform the ways that we keep trying to improve our campus. But you’ve heard all that from me before, and by now you either believe me or you don’t.

Nonetheless . . .

We really need your help in getting first year students to respond to our End of the First Year survey. It’s especially important because we’ve been paying close attention to the experience of various subgroups of students (i.e., African-American students, Hispanic students, first-generation students, students coming from particularly low income families, etc.) that have historically not succeeded at the same rates as more affluent white students. In order to have the most robust data from these students, we need to do everything in our power to encourage participation.

And this leads me to my shameless plea.

Please, please, please: if you interact with first year students or have the wherewithal to communicate with first year students, would you please take 30 seconds to make a personal plea on behalf of the college and encourage them to complete the first year survey? All first year students received an email earlier today inviting them to take the survey. I’ll send a link to anyone who would like so that they can include it on their course’s Moodle site or web page.

Thanks very much. It really does make a difference.

Make it a good day,

Mark

And it’s down to three . . .

Good morning everyone!

It’s not every week that you get to see three pretty smart people talk about the way that they might approach a leadership role as provost at Augustana College.  So if you can find a way to be there, I hope you’ll come to see each of the provost candidates present this week.

Each of them will be presenting at 11 AM and at 3 PM on Monday, Tuesday, and Wednesday, respectively. They will each be giving the same presentation in the afternoon that they gave in the morning, so you can come to one or the other.

Your participation in this process matters for several reasons.

  1. The more feedback the better for the search committee after all three finalists have been to campus.
  2. The more questions asked of the candidates the more everyone in attendance gets a sense of each candidate’s approach to public communication.
  3. The more people in attendance at these presentations the more we communicate to each candidate our investment in our provost and the college.

So come on down to the Wilson Center as often as you can make it.

Make it a good day,

Mark

What do you do when change finds you?

Welcome back from the short holiday break. Don’t tell the wellness folks, but I hope you got to eat all the pumpkin pie and whipped cream you could stand!

But just in case you thought that you were going to ease your way back into the comforting routine of winter term, I thought that now would be the perfect time to tell you about a nifty change that is coming down the pike.

What if you could see your IDEA feedback summary and student comments as soon as you submitted your grades for the term? It seems to me like that would be pretty awesome.

And what if you could get real-time IDEA feedback from your students in the middle of the term so that you could adjust on the fly? It seems to me like that would be pretty cool, too.

Remember a year or two ago when I reported that IDEA was going to phase out their paper forms some time in the next several years? Well, I’ve been informed that the paper forms will breathe their last collective breath in the spring of 2018. That means that, unless we want to go out onto the market and audition all of the other players in the course feedback survey industry (from whom I get phone calls or emails at least twice a week about their “exciting fully customizable online format”) or take on the monumental task of building a homegrown course feedback system, we need to plan on moving to a paperless IDEA system in the fall of 2018.

To be honest, I may have oversold this change a bit. In reality, it’s not going to change the daily life of an instructor much. You’ll still choose your learning objectives at the beginning of the term, and the students still complete the same set of items (albeit with some improvements that actually align better with our own college outcomes) at the end of the term. In many cases, you’ll likely opt to use class time for students to enter their responses on their phones, tablets, or laptops instead of coloring in little circles on a piece of paper. Solving the potential problem that some students won’t have a device that can access the survey is probably a pretty simple one (one possibility would be to borrow a neighbor’s phone or laptop).

Now I suspect that some of you have questions about how this is going to work. And we wouldn’t be doing our jobs if we didn’t wonder about all of the ways that the paperless process could go horribly awry. But after quizzing the IDEA consultant, it seems as if they’ve found a myriad of ways to avoid the obstacles we most often worry about (e.g., low response rates, satisficing, etc.).

Nonetheless, we will host plenty of opportunities to answer questions about the details of this change – both in person and online. So what do we do when change finds us? Put your arms out wide and embrace the possibilities!

Seriously, what else are you going to do?

Make it a good day,

Mark

 

Reading my way out of a sleepless weekend

Good morning, everybody!

That greeting is intentionally more peppy than I feel today.  Sometimes I have to try to con myself into a better place.  Although I’m not sure what to do with the fact that this approach actually works for me, today I don’t have the energy to quibble with the ends justifying the means.

The past few weeks at Augustana have been hard to watch. Getting ourselves to a deeper understanding of difference and how to communicate with each other despite those differences sometimes seems simultaneous deceptively obvious and painfully impossible.  I’d love to whip out some perfect data point and, with an magician’s “abracadabra,” cast a healing rainbow of glitter across the campus. But reality is never presto-change-o with a dollop of whip cream.

So instead, I’m going to share links to a couple of articles that gave me just a little bit of an uplift this morning.  I think there is something in both pieces that is worth pondering. Each of these articles were in the Chronicle of Higher Education – the first one just today and the second one earlier in the summer.

A Gorilla-Masked Student’s Attempt to Provoke is Met with Peace

Talking Over the Racial Divide

Sometimes you gotta “make it a good day,”

Mark

I so wish I had written this!

Hi Folks,

Yes, I’m late with my blog this week. And I’m sorry about that. But I’ve been busy thinking about ways to organize my desk. And that’s something.

Brian Leech shared this with me yesterday, so he deserves whatever credit someone is supposed to get when they share something with someone who then “borrows” it to present to his blog audience in place of something that actually required original work. So all thanks goes to Brian for enabling my slacker gene this week.

We all need to laugh at ourselves and the absurd parts of our work sometimes. So enjoy having a “go” at the assessment culture run amok and the weird world of Institutional Research.

RUBRIC FOR THE RUBRIC CONCERNING STUDENTS’ CORE EDUCATIONAL COMPETENCY IN READING THINGS IN BOOKS AND WRITING ABOUT THEM.

From Timothy McSweeney’s Internet Tendency blog.

Make it a great day!

Mark

So how do our retention numbers look now?

Early in the winter term, I wrote about the usefulness of tracking term-to-term retention. This approach is particularly valuable in evaluating and improving our efforts with first-year students, since they are the ones most susceptible to the challenges of transitioning to college and for whom many of our retention programs are designed. Now that we have final enrollment numbers for the spring term, let’s have a look at our term-to-term retention rates over the last five years and see if our increased student success efforts might be showing up in the numbers.

Here are the last five years of fall-to-winter retention rates for the first-year cohort.

  • 2011 – 94.1%
  • 2012 – 95.6%
  • 2013 – 97.0%
  • 2014 – 95.9%
  • 2015 – 96.6%

As you can see, we’ve improve by 2.5 percentage points over the last five years. This turns out to be real money, since a 2.5% increase in the number of first-year students returning for the winter term means that we retained an additional 17 students and added roughly $84,000 in revenue (assuming we use 3-year averages for the incoming class and the first-year net tuition revenue per term: 675 students and $4940, respectively).

But one of the difficult issues with retention is that success is sometimes fleeting. In other words, retaining a student for one additional term might just delay the inevitable. Furthermore, in the case of first-year term-to-term retention the fall-to-winter retention rates can be deceiving because we don’t impose academic suspensions on first-year students after the fall term. Thus students who are in serious academic trouble might just hang on for one more term even though there is little reason to think that they might turn things around. Likewise, students who are struggling to find a niche at Augustana may begrudgingly come back for one more term even though they are virtually sure that this place isn’t the right fit. With that in mind, looking at our fall-to-spring retention rates would give us a more meaningful first glimpse at the degree to which our retention efforts are translating into a sustained impact. If the fall-to-winter retention rates are nothing more than a mirage, then the fall-to-spring retention rates would remain unchanged over the same five year period. Conversely, if our efforts are bearing real fruit then the fall-to-spring retention rates ought to reflect a similar trend of improvement.

Here are the last five years of fall-to-spring retention rates for the first-year cohort.

  • 2011 – 92.1%
  • 2012 – 93.1%
  • 2013 – 93.3%
  • 2014 – 93.5%
  • 2015 – 94.1%

As you can see, it appears that the improving fall-to-winter retention rate largely carries through to the spring term. That translates into more real money: approximately $69,100 in additional spring term revenue. Overall, that’s about $153,000 that we wouldn’t have seen in this year’s revenue column had we not improve our term-to-term retention rates among first-year students.

Certainly this doesn’t mean that we should rest on our laurels. Even though retaining a student to the second year gets them over the biggest hump in terms of the likelihood of departure, it still seems to me like small consolation if that student doesn’t ultimately graduate from Augustana. However, especially facing the financial challenges that the state of Illinois has dumped in our lap, we ought to pat each other on the back for a moment and take some credit for our work to help first-year students succeed at Augustana. The data suggests that our hard work is paying off.

Make it a good day,

Mark

No matter how bad you have it . . .

Hi Everyone,

I suspect some of you are still scarred from my many requests for syllabuses in preparation for the HLC visit. I won’t soon forget the enormity of that project.

Yet I am outdone once again by a news report of a syllabus collecting project that makes my little effort seem like a game of pick-up sticks.

Thanks to Cyrus Zargar for sharing this with me – even if I am a little concerned by how quickly my name came to mind!

“What a Million Syllabuses Can Teach Us”

Ugh.

Make it a good day,

Mark

 

A short post for a short week!

Its hard not to think that Thanksgiving might need a better lobbyist. Every year the warm anticipation of the holiday seems to get overrun by Christmas decoration sales that start the day after Halloween, and then drowned out by the deafening onslaught of Black Friday advertising during the week of the actual holiday.

I’m not looking to start a movement, but over the weekend I decided to take back my own Thanksgiving. So from my little corner of the world, I want thank all of you, my friends and family at Augustana College. Thanks for inviting the Office of Institutional Research into your conversations, valuing our observations, and utilizing our insights. Thanks for trusting us with all of your data and for making us feel like genuine contributors to the life of the college.

I hope each of you get the chance to find a moment of peace on Thanksgiving.

Make it a good day,

Mark

A casual and incomplete FAQ for our current employee survey

Even though this is my fifth year at Augustana, the concept of Muesday still throws me for a loop. Maybe this is because I don’t have to think about it much, counting beans in my little office all day every day like I do. Conversely, most faculty I know talk about it as if it’s the most normal concept in the world, no matter if they’ve taught at Augustana for a couple of years or a couple of decades. And even though I think I’ve developed a failsafe cover to hide my ignorance (toss my head back, laugh, lean in while I bat the air in front of my face, say emphatically, “of course, what was I thinking!” while rolling my eyes), it’s an annual reminder for me that the concepts each of us take for granted aren’t always so obvious to everyone else.

I’ve been reminded of this reality again as I’ve been inviting everyone to fill out current Augustana College Employee Survey.  More than a few people have expressed concerns about anonymity and confidentiality.  A few have even floated impressive conspiracy theories of NSA-caliber data scrubbing.  So before I have to run off to my weekly administrator neural network reprogramming and empathy reduction session, I thought that I’d try to answer the anonymity and confidentiality questions in a little more detail. (Yes, I’m kidding. The administrator neural network reprogramming and empathy reduction sessions are every OTHER week and don’t meet this week because it’s MUESDAY!)

When I promise anonymity to everyone who responses to the Augustana College Employee Survey, that means that I don’t ask for your name or other information that directly identifies you. It also means that the software doesn’t collect your Augustana user ID or the IP address of the computer that you used to complete the survey. In order to do this, I turn off a setting in the Google Forms software that would normally add this information to the dataset.

Turning this feature off also means that the survey is publicly accessible – a potential downside to be sure. So it is technically possible that each of the 340+ survey responses I’ve received aren’t actually coming from Augustana employees. But that would mean that somebody somewhere else has acquired the web address of the survey and has spent their days and nights repeatedly filling the survey out over and over with just enough variation of answer choices to avoid suspicion.  Yeah, I doubt it.

Some folks have pointed out that there are enough demographic questions that there might be a way to identify some respondents. This is technically true: if someone had access to both the college’s employee database and the current employee survey dataset, one could probably figure out a way to be pretty sure about the identify of some of the respondents, particularly if one were to triangulate several demographic characteristics (e.g., race and age data) to pick out subgroups of employees that have only a few members. Of course, the only person on campus who has access to both of these datasets is, well, me. If you think that this is a likely explanation for how I spend my time … I guess I sort of doubt that you are even reading this post. Nonetheless, to be clear – I’m not trying to figure out what you said in your survey. And I’m not taking that information and slipping it under someone else’s door so that they can hire henchmen to come to your office and hide your keys. It’s not that I don’t care.  I’m just too busy.

All joking aside, this survey does ask some questions that can easily be perceived as risky to answer. So, if you are concerned about anonymity but want to respond to the survey, just leave any demographic question that cuts too close to the quick blank.  That way you don’t have to worry about having your anonymity violated. I think we’d rather be able to stir your opinion into the mix even if it might not get included in more complex analysis.

Confidentiality is a little different from anonymity. There are numerous student surveys where we promise confidentiality but not anonymity. We often will ask students for their ID number so that we can merge the data they provide with prior institutional data so that we can take a longer view of our students’ four years at Augie, looking for patterns across the entirety of their college experience. Confidentiality specifically refers to how we will share any of our survey findings. When I promise confidentiality, I am promising that I won’t share the data in any way that might link your set of responses to you. Instead, all data findings will be shared as averages of groups, whether that be the entire group of respondents or small subgroups of respondents.

This does again raise the question that some have asked about protecting the anonymity and confidentiality of those who are members of sparsely populated subgroups. When I promise confidentiality, I have to also consider the possibility that presenting data in all of the ways that it can be sliced and diced could lead to violating someone’s confidentiality. To allay this concern, I am ensuring confidentiality by simply not sharing any results in a way that might allow folks to reasonably infer any individual’s responses. I will not share any average responses to questions where the number of respondents in that particular subgroup is less than five. This makes it much less likely that anyone could determine the nature of someone’s individual responses based on the average responses from any particular subgroup of responses. So, for example, if we have less than five respondents in the category of employees who have worked here between six and ten years, then we won’t share any results for any question by the number of years employees have worked at Augustana.

Just like the anonymity question above, if you are worried that your confidentiality will be violated, don’t provide answers to those specific questions.

Even though we have already received many responses to this survey, we still need many more because the more that we have, the more likely it will be that we can look at subgroups of responses and analyze this data without violating anonymity and confidentiality.

Getting better as an organization is hard work. At its core, it requires that we all put something into it.  Completing this survey is a big first step. I hope you’ll all give it a shot.

Make it a good day,

Mark