You know how there’s all those jokes about trying to get on the tenure track, trying to stay on it and actually making it through review? You know, all those stories about the hell that is trying to survive and get by in academia where some of it seems like an undocumented feature? Well, this week I had a day where I felt like I was acting out a metaphor for academic life. All I had to do was find a room.
There was a seminar. A networking session. Free pizza. An opportunity to meet new
drinking buddies peers who could assist my slow climb to world domination the next boulder up the academic mountain. The instructions were simple enough; go to room 747, in the Health Sciences building, at 12. Simples. The building is logical enough (I thought), rooms number 100 – 199 are on the first floor, rooms 200+ on the second, etc. Pretty straight forward. I’ve never been to the 700s before though, so I go early incase I happen to get lost.
I get to the building excited at the possibility of meeting more like-minded people and I find there are only 6 floors. It’s either the 6th floor or the roof. I pick 6th. This is when I learn that rabbit warrens have been built within our university buildings. There are corridors hidden within corridors. Some seem to shut themselves off from others once you are in them. Some corridors look like you shouldn’t be there, either that or they ran out of money for paint and adequate lighting.
I figure I’ll handle this logically. If there is no 7th floor then the 700s should be somewhere here, a split level floor perhaps? So I start walking in the direction where the 600 numbers increase. I trudge onwards, turning corners into more corners. No one else appeared to be here. I saw cadavers. No seminar room. Still alone. Is the 6th floor out of bounds and no one told me? The 600s started decreasing. I took a door that linked me to more corridors that enclosed the corridors I’d been wandering down. Numbers started increasing and then I found the 700s… they increased, for a bit. Then they stopped. The 600s started again. This madness continues for a while as the inclination to keep looking over my shoulder grows. I start to wonder if this is some hidden camera show for tenured faculty to watch. There is no room 747. Tenured faculty send out those emails and then watch the rest of us come and visit their maze of hell.
But, I did check… and there was a room 747 listed on the university webpage. Having met professors who have eschewed excel files for a print out of a graph so they can check a line of fit is straight or asked me to weigh cutouts of my peaks instead of using a computer to integrate them – talk about old school – I can’t imagine tenured professors would have the know how to hack the university web just for a joke.
Then it dawned on me. These corridors don’t lead to room 747. It was obvious. I work at Hogwarts! I’ve overlooked the possibility that the staircases change! It’s just a matter of waiting for the staircase to move so I can gain access to room 747. But then… which staircase? It appears this may be a group only for those fledgling faculty who are in “the know”. If you can find it, you can move your piece forward in the game of tenure. I have not yet received the staircase knowledge. Must I go back to “Go”? Do I collect $200?
Or, is it like the room of requirement? Where, if I had wanted it hard enough, the magical room 747 would have appeared to serve my needs. I thought I really wanted this career, but the door didn’t appear when I hovered by the other 700s. Maybe I don’t want to be a professor as much as I thought? But then, they did have free pizza. And, I really did want that pizza.
I’m an academic. In search of a room.
I’ve been playing around with the format of our weekly journal club. The weekly activity now alternates between critiquing the draft manuscript of a group member and the published works of others. The motivation for this format is to engage the students more by making clearer links between reviewing others work to strengthening their own writing, as well as keeping us up-to-date with research outside of the group.
Having worked in industry, I’m very passionate about ensuring any students I mentor are taught skills that they will need in business as well as helping them with their academic work. Hence, I decided to introduce an often used business procedure to our journal club, the SWOT report. A SWOT report is a short document that succinctly summarizes the strengths, weakness, opportunities, threats (and sometimes also trends) of a project. It can be used in many contexts, from assessing human resource requirements to assessing whether a new marketing plan needs to be adopted.
T: Threats and Trends
I know many of my students won’t have heard of SWOT analysis or reports, yet I know it’s likely they will have to write one if they progress to a management level outside of academia. I also know that some of these techniques used in business are used precisely because of their effectiveness for assessing information.
So, how can the SWOT report be utilized in an academic journal club setting and what are the advantages?
It helps when critiquing literature to have a structure to how you are going to group your various thoughts about what is good and bad about an article, as well as where you see the work positioned within the field and the future prospects for that avenue of research or the approach being employed.
When reviewing literature the last thing you want is to increase the time involved by writing an in-depth report. Writing such a report is not what the SWOT analysis is about. The SWOT approach can be harnessed to help quickly digest a research article in a structured manner by using it as a method to create a one page summary critique.
1. Separate a single letter or A4 sized piece of paper into four quarters and write at the top of each quadrant the letters S, W, O and T.
2. As you read through the paper try to identify both strengths and weaknesses and start entering them as bullet points in the relevant quarter of your page.
Questions to keep in mind as you do this are:
- Is there something you think is missing, such as an analysis you deem essential to be able to draw their conclusions
- Were the results significant?
- Were the statistical tests and analysis used appropriate?
- Was the paper written in a clear and understandable way?
- Does the study contribute to the body of knowledge?
- Is the approach used, experimental or theoretical, appropriate for this work?
- Is the work reliable, in your opinion?
- Based on the methods and analysis used, do you think the results are valid?
- Are the methods well described?
- Could the work be reproduced from the information given?
3. Now identify opportunities. Re-scan the article if you need to.
Consider the following:
- How could the research be extended?
- What would you do next if this were your research?
- Was there a set of results that you think warrant further examination/confirmation in a follow up paper?
- Does the paper reveal something new, a new approach or hypothesis, which could open a new direction of research and perhaps be the start of a new field with lots of opportunities for future research?
- Do you see an opportunity for a study to try and refute the conclusions?
4. The last step is to identify threats and trends. This is a little bit different from what we’d consider direct weaknesses of a paper and more about considering how the work presented sits within the field.
Trends can be the increasingly common use of an experimental method for examining a certain property, or the popularity of a certain area of research, such as that illustrated by the sudden growth in research articles regarding intrinsically disordered proteins.
Threats are usually aspects of the work in opposition to those that you would list under “opportunities.” For example, it could be that the researchers have used an approach about to be superseded by more modern techniques. It could be that the authors have used experimental methods with known limitations that could bring the results of the article into question when more advanced experiments become available.
Threats could also be other groups that are known to be performing the same (or similar) research, possibly with different results that conflict with the conclusions made in the article. This would lead us to consider if both sets of results are valid and whether real controversy exists or whether it is a case of one group being limited to using methods that may not report on the properties being measured accurately enough.
Other questions to consider are:
- Is the research methodology following old trends?
- Are they introducing new methods?
- Or using methods, recently introduced by others, that seems to becoming a popular approach in the field over other older methods?
- Is the research presenting anything that builds on current knowledge or is it clear that this research avenue is coming to a close?
- Have the researchers taken advantage of recent advancements in experimental or theoretical methods?
- Is the continuance of this line of research threatened by a lack of resources?
- Are the methods they used soon to be superseded by advancements that might bring their results into question?
Here I am! I’m here!! Finally!!!!
After exploding and malting some weeks ago… Bench Monkee is finally ready and filming begins.
The moment you’ve all been waiting for… your host is here! Let me introduce, for the first time, Bench Monkee.
You know when you have one of those days where your puppet is just about ready to start filming and you’re all excited that you might get to upload something to YouTube, but then stuffing explodes out of the side of the monkee’s face? I just had two of these sorts of days, each involving unpicking, de-stuffing, re-sewing, re-stuffing and a lot of cursing. To add insult to injury, I have made the most terrible decision in regard to the stuffing I selected. If you ever make a puppet or teddy bear yourself, do not use SynFeather stuffing. I ordered it because it’s supposed to be great for creating firm shapes but it sticks everywhere! It’s migrated to every room, it’s in my bed, in my pants… everywhere. My house currently looks like a giant with a bad case of dandruff danced by. I even got home from work yesterday, which was a day in itself where my workplace was surrounded by SWAT teams (see here and here), only to embrace my partner and find that he and his beard were also covered in this stuff.
Since I will be moving in four days and am currently surrounded by empty boxes that need filling with everything I own, now also covered in the stuffing that will never go away, I will not be uploading anything to YouTube yet. I am so sorry for those of you I know were looking forward to seeing the puppet and checking out my channel. I will be back in touch once I’ve settled down at the new pad and I am no longer being tormented by my ill choice of stuffing.
Until then, take care and beware any white synthetic stuffing, it looks friendly but it will suffocate you while you sleep.
Things have been frantic here, but me and SZ found an apartment and are moving in a weeks time. It’s all happened so fast that I’m afraid my Bench Monkee activities were neglected. Apologies for not posting something today, as promised. I will get something to you this weekend though as I just finished the assembly of the puppet 🙂
Hope you are all having a great weekend!
So, Sequester Zone mentions me on YouTube and my readership stats go crazy. Thank you all for coming to visit my blog and check out what this Bench Monkee thing is all about.
As SZ mentioned in his video (The Mismeasure of Man Criticisms), we are having landlord issues at the moment so my original plans to launch on March 28th have been sidetracked. I will be trying to upload something on that day, unfortunately not the episode I’d originally planned, which was going to be about the scientific method and recent discussions that have occurred about retiring certain concepts, such as falsifiability. One of my previous entries was about this, if that sort of thing interests you. So, although the episode I’d planned will not be ready, I will upload something, even if all I get to do is a small intro vid so you guys can get to meet Bench Monkee.
As a reward for stopping by my blog, I give you pictures… of severed limbs and eye balls… eek! Actually, this pile of hairy body parts is Bench Monkee, pre-assembly, who will be the superstar of the channel (one hopes). Hope to be entertaining you soon on YouTube, until then, ttfn. :(|)
I jest. Although it is possible that somewhere on this planet Karl Popper’s spirit may be torturing those scientists who believe that the idea of falsifiability is a dead concept and should be retired, it is highly unlikely. I do often fantasize that Karl Popper is regularly haunting Sean Carroll after reading his article on Edge.org. I do.
I know you’re probably wondering what I’m rabbiting on about. I’m about to launch into a discussion about falsifiability and convince you that it is critical to performing good science.
Now, I know that many of you might not have come across this idea of falsifiability, at least not in the scientific context so let me briefly explain.
capable of being tested and falsified by experiment or observation
The way the scientific method works is that we do an initial observation of something, do a bit of wondering and then formulate a hypothesis and it’s opposite, the null hypothesis. These statements are of a form that is testable and thereby falsifiable. For example, if I thought the world was shaped like a pineapple my hypothesis would be:
Hypothesis: Earth is shaped like a pineapple.
The null hypothesis would be: Earth is not shaped like a pineapple.
These are two statements that can be tested and are capable of being falsified, i.e. shown not to be true. Obviously, measurements of the circumference of Earth at different points has been done confirming that the Earth is not pineapple shaped.
So, once we have a testable statement, we then do experiments to determine if our hypothesis was correct or not. The best way to test a hypothesis is to try and disprove it, usually by applying the null hypothesis. Then based on our results we decide whether to accept or reject the null hypothesis. Basically, this is done because it is a stronger and more certain test of a hypothesis to look for situations where it is untrue. Oftentimes, because we’re human, we have tendencies to focus on or accept evidence that backs up our beliefs, ideas and hypotheses. This is called conformation bias. So, in order that hypotheses be testable they must be falsifiable and it is often best to start by looking for evidence that your hypothesis is wrong. By doing this you’re framing your idea in a way to try to disprove, and thereby falsify, your initial hypothesis.
So here’s the issue I have. Some people think the concept of falsifiability should be retired from science. I may only be a monkee, but this is the dumbest thing I have heard in a long time. I do believe that, if such a thing were possible, Karl Popper would be turning in his grave if he knew about this. (Karl Popper was a philosopher of science who was all about empirical falsification).
Let me clarify what is wrong with rejecting the concept of falsifiability. Carroll claims that some things can not be falsified (see the Edge.org link at the start of this article). He is completely correct that some things, at this time, cannot be falsified. In his essay about this he mentions predictions within the field of physics not matching observation and the possibility of multi-verses being the explanation for these discrepancies. Now, just because predictions, hypotheses and theories cannot be confirmed or tested completely now, doesn’t mean there won’t become a time when they can. The hypotheses/theories/predictions Carroll mentions have inherent falsifiability, it’s just really difficult to perform this falsification at this point in time.
The gap between what we can conceive and what we can confirm is not a good enough reason to remove the concept of falsifiability from science. Carroll doesn’t seem to be able to see that in order to be confirmed scientifically hypotheses and theories about this phenomenon must be falsifiable. His argument stems from him being unable to see beyond the current state of scientific observation and see that the overall philosophy of falsifiability still applies, even if it can’t be applied now. Methodological development is not static. Perhaps one day we will be able to observe and measure multi-verses? However, Sean Carroll thinks that because the technology doesn’t exist to falsify some theories, hypotheses and predictions in our time we should throw it out? No! Absolutely not! People like Sean Carroll need to realize that while certain theories can’t be falsified at this current time, it doesn’t mean that they are not falsifiable or that the philosophy of falsifiability is no longer crucial within science.
Lack of the methods to falsify now does not mean lack of future falsifiability!
It is a testament to man’s intelligence that we often formulate ideas and theories beyond our time, beyond our current methodological capabilities. The scientific method doesn’t need to be changed, some scientists just have to learn to be patient and realize that certain hypotheses may not be testable for some time, maybe not even in their lifetimes. Just because a hypothesis isn’t testable now, doesn’t mean it won’t be testable in the future and certainly does not negate the continued use of the concept of falsifiability.
Please help me save the concept of falsifiability. If you ever here someone suggesting we should retire the idea of falsifiability from science please re-educate them and perhaps gently suggest they should consider another career.