Showing posts with label evaluation. Show all posts
Showing posts with label evaluation. Show all posts

Sunday, March 6, 2022

Listen, don't listen

 


Academics operate in a world where the rules of the game are not fully known to the players. In addition, they are in a state of constant change. This is because the rule book is what academics in temporary positions of power choose to do with it. Do they choose to reward monographs in Swedish or international peer-reviewed journal articles? Do they count the number of publications or do they read them and try to form their own opinion? Do publication channels play a role or not? Do co-written texts make you more or less competitive?

As if that wasn’t enough, the playing field and the players are constantly changing. In the 1990s, when the Swedish higher education sector was expanded, some historians went straight from receiving their PhD to having a permanent position as senior lecturer. This is completely unthinkable today. In the 1980s, it was unusual for academics to apply for positions at universities other than the one where they received their PhD. Today, academics from elite American universities may apply for a position as associate professor in Kristiansand in Norway. At Swedish universities, foreign leading academics are still rare in the competition. However, the Department of Archaeology, Conservation and History at the University of Oslo has experienced a rapid and dramatic shift in the 2010s. Here, local applicants have not even bothered applying. For the first time in history, a whole generation of Oslo historians have not had a real chance of being employed at their home department.

All these transformations make it difficult to get good advice. Nevertheless, many of us are pretty quick to offer such advice. “Only peer-reviewed articles count,” “edited volumes don’t make you competitive,” “without a second book, you’re nothing,” “no one cares about premodern history.” Nevertheless. As soon as expert opinions in relation to a position are made available, you realize that the rules of the game are hardly set in stone. Above all, you realize that none of the players knows for sure how the rules will be interpreted by the referees. What is crucial for one research funding body is less important for another. A behavior praised by one professor is seen as worthless by another.

At the same time, however, not everything is entirely arbitrary. There are patterns in the noise. If we look at longer time series, this variance tends to level out. Good academics often find their place sooner or later. But out of everyone prepared to offer advice – who should you actually listen to? How does the opinion of a renowned professor hold up against that of a recent associate professor or a successful postdoc? And to what extent can and should you think strategically? Isn’t the research profession primarily based on passion? Doesn’t following your heart result in the best research?

My general principle is to listen to many people but to be careful when it comes to trusting someone blindly. I look upon being absolutely certain how the academic world works as an alarm bell. I simply don’t think anyone knows. How could you when the rules of the game, the playing field, the players and the referees are constantly being replaced?

The group I trust the most, however, are the people a few years ahead of me. Perhaps one may talk about this in terms of a “proximity principle.” Sure, a lot of changes are taking place. Yet, the academy is a slow-moving world. The dynamics of the game do not change overnight. What worked yesterday will probably work tomorrow. However, it’s not at all certain that the recipes for success in the 1990s or 2000s are still effective.

Furthermore, at least in my experience, it’s treacherous to compromise too much with your own ideals and research ambitions. Things typically do not end well when someone tries for a long time to do something purely based on strategic reasons. What brings joy and pleasure must also be included. Otherwise, the sweet taste of victory may turn into ash in your mouth. Carrying out a research project that doesn’t engage you at a deeper level is not all that much fun. On the other hand, academics who can’t imagine doing anything else than what they want to do and who hate all this talk about games, strategy and career typically don’t have all that much fun either. This is a situation where bitterness, whining and envy are close at hand. This is something I try to stay clear of as much as possible.

Hence, I advocate adopting a mixture of strategic thinking and scholarly idealism. What you need to do, as Zlatan Ibrahimović says in his autobiography, is to “listen, don’t listen.” If you are to get anywhere in academia, you must be able to take advice and make some concessions. But you also need to dare to go your own way and shut out the counsel of others. Knowing when to do one thing and when to do the other is an art. Perhaps even one of the most important academic skills.

Sunday, February 13, 2022

Assessments and being evaluated





An element of an active academic life consists of assessing others and being assessed. It runs like a common thread from the first exam as an undergraduate to the expert opinions of the retired professor. But how does it actually work? How do you know whether something is better than something else? What is scholarly quality and what makes someone qualified to assess this?

Apart from grading students, I have in the past few years examined PhD program applications, journal articles, research applications and served as external examiner. I have not received any formal training in assessing and evaluating historical scholarship. Or perhaps I have. I have read, analyzed and evaluated scholarly texts since at least the early undergraduate level. I have participated in critical discussions around seminar tables. What’s this if not formal training in scholarly evaluation?

The seminar culture in which I was trained reached a new level when I was accepted to the PhD program. Sure, the PhD program contained much coursework. However, the higher seminar of the Department of History is where I learned the most. In this setting, you were expected to be present and say things – even though the seminar didn’t offer any credits or you were formally obligated to attend. In my PhD generation, there was a strong sense of loyalty toward the seminar. Being there was more important than sitting in front of the computer writing or in the archive digging.

Then as now, I greatly appreciated general seminar skills. I particularly appreciated those who didn’t always repeat the same thing over and over and who were also capable of giving good comments on things outside their own area of expertise. This requires both intellectual flexibility and an ability to perform empathetic readings of texts that don’t really interest you. In order to do so, you need to approach texts on their own terms and be able to distance yourself from your own worldview. Being able to think on your feet doesn’t hurt either.

These are all typical “invisible skills.” They don’t appear in publication lists, CVs or portfolios over teaching experience. They may, however, sometimes be seen between the lines in books and articles. But far from always. One is often surprised by how unremarkable renowned academics can be when operating outside their own comfort zone. On other occasions, you are amazed at how skillfully a master’s student discusses something he or she doesn’t know all that much about.

Somewhere here is my basic view on what constitutes scholarly quality. Perhaps one could specify it as a kind of intellectual elasticity and a general scholarly ability to assess and evaluate things. Over the years, I have come to realize that this is only one basic view among many. Other academics value completely different things the most: creative choices of subject matter, vast and hard-to-access archival materials, ability to write and how to engage in certain types of theoretical reasoning. Some academics value things close to what they do highly. Others adopt the opposite perspective. They expect even more from academics operating in proximity to their own field.

This insight into this kind of value pluralism has over time made me more humble with regard to my own basic view. How can I be sure that it’s all that good? And, by the way, who’s to say that all academics should be good at the same things? The field of history wouldn’t have been particularly interesting if it only consisted of masters of seminars. We also need academics digging in archives and others driven by theoretical curiosity. We need excellent writers and people who can count. Without the diversity of different types of academics, the seminar culture would quickly turn into some sort of self-referential intellectual exercise. An echo chamber.

Another way of thinking about evaluations and assessments is to actually study this phenomenon. People outside the academy might perceive this as the pinnacle of navel-gazing. But for those of us who work in the academy, this is extremely interesting. My own favorite study in this field is How Professors Think (2009) by sociologist Michèle Lamont. In this book, she carries out ethnographic studies on a number of multidisciplinary research councils distributing prestigious grants. She participates in their meetings and interviews the evaluators before and after. She is particularly interested in how professors assess each other’s efforts as evaluators. What brings respect? What makes you trust someone’s judgment? What makes you lose confidence in someone?

This post is not the place to summarize her findings. But the book is definitely worth reading in its entirety. It’s a great qualitative study offering many general insights. It also gives the reader a chance to be a fly on the wall in one of the rooms where your own applications actually end up. Because even if you only see acceptance letters, rejection letters and, sometimes, the arguments made for or against you, these decisions are obviously made by people. But they are not made individually but in groups. A bit like a seminar.


Further reading: "The role of writing applications in the research process" and "What academics can learn from players"

---------
Do you want to sign up for the blogs mailing-list? Send an e-mail to david.larsson_heidenblad@hist.lu.se

Sunday, June 13, 2021

Stopping


It’s immediately clear to everyone when the spring semester has come to an end. The corridors are emptied of colleagues. There are fewer people in the breakroom and there are no longer any students around. As for me, I still have three weeks left to work. But this is the last blog post of the semester. 

At this particular point during the year, I have a habit of evaluating the previous semester. My focus is not primarily on objectives and results. Rather, I try to get a sense of the feelings I’ve had throughout the semester. What has been the most enjoyable to do? What has been the greatest learning experience? What has given me energy – and what drained me of the same? 

The first thing I do is to simply take a blank sheet of paper and start writing what comes up in my head. Good as well as bad things. What I try to get on paper in particular is what I have fully engaged in. Which tasks actually make me forget about time and space? Which contexts and people inspire and stimulate me? However, it’s just as important to write down what’s been hard. Perhaps resulted in concerns and anxiety. All in all, such an unstructured piece of paper gives me an overview. It becomes a map over emotions. 

After having written such a piece of paper, it is time to start thinking. What of all this do I want to spend more time on? How should I go about this? What do I want to avoid or minimize? How do I do that? These are simple questions and a simple process. But if you take it seriously, it takes a fair amount of time. In my case, I need a couple of days. As a matter of fact, I have set aside the last week of the semester to evaluate and plan. In my experience, the latter is not meaningful unless you have done the former. It’s like groping in the dark. How can you plan for the future if you have no idea what you want it to look like? (Or don’t want it to look like!) 

In essence, my own spring semester has been fun and rewarding. I think it has a lot to do with this blog. Sure, at times, it has been scary writing about sensitive and personal things, but my blogging has always felt meaningful and important. Publish and be damned, as journalists say. The input from you readers has also strengthened my impression that there is actually a need for this blog. There are too many things we don’t discuss with one another. And far too much knowledge is tacit or silent. If this blog is able to make an ever so small contribution in this regard, I’ll be more than satisfied. 

In professional terms, the highlight of this semester was my debut as an external examiner. This occurred in April when Pär Wikman defended his thesis Kulturgeografin tar plats i välfärdsstaten. Preparing for this examination was incredibly enlightening. Carrying it out was exciting. This was the real deal. That is what I want the academy to be like. 

As for my own writing, it has progressed well overall. I have (soon) done what I set out to do at the beginning of the semester. A bit more than three chapters have been written, applications have been submitted and I have finished my review of Wikman’s thesis. In addition, Johan Östling and I have written a so-called position paper on the history of knowledge, which I hope will be widely discussed. 

But, as I have hinted in previous posts, there was a downside to this semester. My schedule has at times felt tight. The room for things going sideways and for having spontaneous ideas has been limited. Things have worked themselves out, but I want greater margins in my life. Because if there is one thing I have learned in recent years, it’s that things will happen. You just don’t know what and when. A sustainable plan must take this into account. Without a generous buffer, the fun aspects will not be as fun. In a worstcase scenario, they will just be difficult. And in such a case, something is wrong. 

What I’m trying to say here is thus that I should schedule more time for my book writing. I haven’t finished my detailed planning for the fall, but I will probably aim to write two chapters instead of four. If I do this, there is plenty of time for the other writing and work tasks I have already planned. In practice, furthermore, it doesn’t matter whether the last two chapters are written next spring. It requires a bit of planning in terms of when I use certain funds for certain projects. But that is certainly something that can be worked out. 

The alternative would have been to cut down on various commitments or write a shorter book. Neither feels right. The things I have agreed to do are things I want to do and look forward to. And as far as the length of the book is concerned, all the chapters I have planned are important for the whole picture. They serve unique functions and dropping one or two of these would result in a different book. This would not necessarily be a bad thing, and if the circumstances had been more pressing, I could have made compromises. But academics don’t write books all that frequently. For me, it will probably take another five years before I start thinking about doing it again. That’s why it feels important to stick to my vision. 

Finally, I would also like to take this opportunity to thank you for this semester. Having so many interested and encouraging readers has been a privilege. I hope that you’re not yet tired, because there is still a lot of blogging to be done. I’ll get back to work in September. Have a nice summer!

---------
Do you want to sign up for the blogs mailing-list? Send an e-mail to david.larsson_heidenblad@hist.lu.se