FEC Reflections, 2013 Dr. Jonathan Schaeffer, Dean of Science Dr. Arno Berger, Mathematical and Statistical Sciences Dr. Chris Cairo, Chemistry Dr. Duane Froese, Earth and Atmospheric Sciences
1
2
Arno Berger (Mathematical and Statistical Sciences) Chris Cairo (Chemistry) Duane Froese (Earth and Atmospheric Sciences) As the three new members of FEC, we have been asked to reflect on our experience for other members of the faculty. Here, we try to capture something of the workload, discussions and our overall perceptions from our experience. The process Near the end of November all members of FEC receive four binders – a hefty stack of about 30 lbs. – covering the activities of 273 faculty and 25 FSOs in the Faculty of Science. For the most part, this consists of the annual report and the chair’s recommendation for each case. But FEC also considers cases for tenure and promotion, continuing appointments for FSOs and, at an earlier meeting, sabbatical applications. Overall, this is a daunting task, just in terms of the volume of material to study, made additionally complex by the diversity of research and activities undertaken across the faculty. It is also a humbling task when you consider the remarkable work, service and teaching being carried out by our colleagues. We would all agree that members of FEC take this work seriously, chairs are well-prepared, and members spend a great deal of time in preparation for the FEC week.
More on process… FEC proceeds through the files from the most junior members through senior professors by order of seniority. Each case is considered individually, but only open for discussion should some committee member have a question about the file. If a file is opened, this leads to a vote at the end of the discussion, otherwise the chair’s recommendation stands. As we learned over the last few years, the FEC process has become more efficient as the chairs have been given targets based on historical results from FEC. Arguably, this has the benefit of allowing the department to be assessed against other members of the same department with FEC reserving the right, through the discussion and voting process, to change those recommendations. For the most part, however, FEC tends to agree with the chairs’ recommendations, though we should add that many files generate significant discussion even if the chair’s recommendation is approved eventually. Diverse cultures across (and within) departments FEC is presented with a relatively standard annual report template that might record teaching of a few to several hundred students, research output from a few to more than 100 refereed papers, and service roles which range from non-existent to ‘how do they find time to sleep?’ These differences are especially pronounced between departments, but are also present within departments. As in previous FEC reflections, the biggest challenge for FEC is developing a fair system of merit distribution that recognizes the diverse roles faculty members need to fulfill in terms of teaching, research and service – each sensu lato. There is a sense that FEC acknowledges these diverse roles as being necessary for the University, Faculty and individual departments to function.
Berger, Cairo & Froese
3
4
More about diversity & culture… Coming to FEC we would have expected that research and publishing are the most important metrics – and in a broad sense this is certainly true. If you happen to have a research program that involves publishing many of your papers in high quality venues, have average to ‘good’ teaching and show a level of service deemed reasonable for your career stage, you will probably do well at FEC. But in several cases chairs were effective at conveying the value of specific contributions, or contributions outside the typical metrics that might have been otherwise overlooked by FEC. Some research programs have needs that are not typical across the faculty, e.g. research requiring significant outreach, and interactions with industry or government that may not lead directly to peer-reviewed papers. It was interesting to see how these non-typical paths can still lead to normal or extra incrementation – but be warned that these cases are few and the commitment from these individuals is truly exceptional. Importance of communication The files presented to members of FEC begin with the summary and recommendation of the department chair. The chair’s report provides an overall summary of the categories of teaching, research and service for each individual; helping to calibrate the discussion of the file at FEC. Additionally, during FEC deliberations the department chair is the primary source of any additional information needed to evaluate the file. Therefore, providing information to your chair will assure the crucial information on your case is available during FEC discussions. If you have published in an unusual venue, or had a notable accomplishment for the year that is not apparent from the usual metrics, let your chair know and provide them with details to inform their recommendation.
Berger, Cairo & Froese
More about communicating… Of course, the annual report is the most valuable mode of communication to FEC. Thus you should think carefully about how to best communicate the unique aspects of your contributions over the last year. Don’t forget that the additional data section can be used to provide information that may not be apparent in a simple listing of your publications or presentations. Information about individual contributions on multiauthored papers, the contribution of students, the order of authorship, or even the value of a particular venue are important factors in the FEC evaluation. On publications, be certain to bold the names of students from your group. Although you may be aware of the prestige of an award within your community, all members of FEC may not be – tell us how many are awarded every year, how big is the field of candidates every year, or even the website that lists previous winners. Take advantage of the system’s ability to incorporate footnotes to provide details on specific items in your report. Consider, too, that FEC has many files to review – so additional data (and your report) must be relatively succinct. Provide the highlights that will help the committee weigh your contributions and make a decision on relative merit. Teaching matters Every year, the newly elected FEC members’ reflections emphasize the importance of teaching as part of everyone’s annual performance. This year is no exception. Teaching is important, and considered carefully by FEC during its deliberations. Having seen the truly impressive breadth of teaching within our faculty, we feel that the accurate and consistent evaluation of teaching constitutes one of FEC’s most difficult tasks.
Berger, Cairo & Froese
5
6
Teaching matters … Yes, teaching does matter, but how exactly it matters is not always easy to discern. On the one hand, there is ample evidence that poor teaching adversely affects merit. With less-thansatisfactory teaching, even an otherwise stellar annual report may attract probing questions and may ultimately be awarded fewer increments than would have been awarded if the teaching had been at least average. Disrespect towards students, disorganized classes or labs, irresponsible conduct of exams are all taken very seriously. Also, FEC does not take kindly to claims of long-term improvement of teaching quality or effectiveness if the evidence is weak or inconclusive. On the other hand, the effect that very good or outstanding teaching has on merit is less clear. It is a humbling experience to realize just how many individuals in our faculty deliver, year after year, superb and inspiring teaching that has a lasting impact on generations of students. As a consequence, it is often difficult to differentiate a strong teaching performance from an already very high background level of teaching quality within the faculty. Repeatedly, FEC was drawn into a discussion as to exactly what constitutes “mainstream”, “good” or “outstanding” teaching, and how to adjudicate merit accordingly. In most individual cases, it was felt that numerical data in the form of Universal Student Ratings of Instruction (aka the infamous IDQ scores) alone did not provide sufficient non-ambiguous information to warrant extra merit. Oftentimes overinterpretation of IDQ scores was seen as splitting hairs and therefore should be avoided. As is the case with all other components of your annual report, additional pertinent information helps: clear evidence of improved teaching (compared to your historical record), sustained efforts invested in meaningful teaching development, taking on of an unpopular course and running it with reasonable success, were all viewed favorably. Berger, Cairo & Froese
7
To summarize: Teaching does matter. While poor teaching is invariably seen as problematic, and IDQ scores barely above average will not excite FEC, a strong case of excellent teaching made in your annual report (and backed up by your chair’s recommendation) may make a strong year - great. In closing On the whole, serving on FEC is informative and rewarding, and the knowledge we gained about the diversity of our faculty, their roles and accomplishments is invaluable. It is safe to say that all of us will look at our chairs’ recommendations with a different eye in the future. We take away the point that there are challenges with the FEC system that are inherent to any evaluation process, but overall the system aims to be fair, and strong contributions are recognized. Could it be better? Perhaps, but the additional challenges mostly come from those faculty members whose career paths, or portions of their workload, are atypical yet at the same time provide important roles for the faculty and departments to operate.
Berger, Cairo & Froese
8
Jonathan Schaeffer, FEC Five Years Later I served on FEC in 2005, 2006 and 2007 as Chair of Computing Science. After a four-year term as a Vice Provost, it was a delight to return to the Faculty of Science and participate in FEC again. To read the annual reports of over 300 professors and FSOs is the highlight of my first six months on the job. The Faculty of Science has many people doing outstanding work – research, teaching, and service. The quality and quantity of contributions across the entire Faculty is stunning. My job as Dean is to brag about the Faculty of Science. The annual reports show that I have a lot to brag about. Let’s start by looking at Faculty output. Simple numbers can give you a hint of what an outstanding year 2011-2012 was. Refereed Publications: 1394 (1349 in the previous year) • • • • • • •
Journals: 1054 (986) Conferences: 205 (193) Book Chapters: 78 (51) Other (Paper Abstract, Poster Article, Review, Other/Generic): 57 (119) Non-refereed publications: 411 Books: 9 Patents: 9
Note the strong trend towards more journals, conference, and book chapters. The growth in these numbers is real – the size of the professoriate in 2011-12 is almost exactly what it was in 2010-11. Given that we have 293 professors, this translates to 4.8 refereed publications per person (a paper with multiple Faculty of Science co-authors counts as one paper in the above data).
9
Clearly there is quantity, but what about the quality of our publications? Assessing the quality of a publication venue is a touchy matter. Bibliometrics (such as publication venue impact factor) is one useful measure, but it may not capture the full impact. For example, some papers are targeted towards area-specific journals that reach a (possibly narrow) target audience. These publications may have lower citation counts than venues that reach a wider audience. Despite valid concerns like this, it is a useful exercise to consider where a journal ranks with respect to its peers within an area. Using the impact factor metric, of the 1054 refereed journals they ranked as: • • • • •
Top 10%: Top 10-30%: Top 30-50%: Top 50-100%: Unranked:
27.1% (27.6% in the previous year) 40.7% (37.3%) 14.6% (13.4%) 10.3% (12.0%) 7.2% (9.7%).
It is rewarding to see that most of our publications are in high-quality venues. The above data is essential for me in arguing to Central that the Faculty of Science is producing high-quality research. As we all know, the real impact is the significance of the ideas contained in the papers, and for this we have no effective metric. The above data (supported with additional analysis) allows me to make a compelling case to Central that we are one of the top Faculties of Science in Canada.
Schaeffer
Let me add one more data point. Much is made of the University of British Columbia’s Faculty of Science. By many metrics, they have surged far ahead of us when looking at national metrics. But a closer look reveals the quantity versus quality trade-off. UBC has the quantity – almost 100 more faculty members than at the University of Alberta. That shows up, for example, in total research funding where they have narrowed the gap with the University of Toronto for top NSERC-funded institution. However, I would argue we have quality. University of Alberta Science researchers brought in roughly $260,000 of research funding per person in 2011-12; UBC researchers brought in close to $210,000. Of course, FEC is not just about research: teaching is an important factor in obtaining extra increments. On this metric, again, the professoriate excels. Let me cite just one metric to substantiate this claim. We all know that USRI data has limitations, but when it is to the Faculty of Science’s advantage to use it, I will. One of the USRI questions is “The instructor was excellent.” On this question the average student response was 4.5 out of 5.0. Not only do students support the above statement (4.0), many of them strongly support that statement (5.0). There really isn’t much room to improve this metric. Collectively, we should be proud of the quality of our teaching. I did not compute any statistics on the research/university/community service that members of the Faculty of Science contribute, but rest assured it is impressive. Let me quote from last year’s FEC report by Doug Gingrich (Physics) and J.C. Cahill (Biological Sciences). As elected members serving for the first time on FEC, they wrote that “We had no idea how little service we do, until we saw all the files. Many members of our faculty do truly amazing amounts of service… .” This year is no different. Many faculty members are incredibly generous of their time – both internally (department, faculty, and university) and externally (research agencies, professional associations, journals, conferences, and outreach).
Schaeffer
10
11
What does the above discussion mean? Compared to my experience on FEC at 2007, it is clear that the performance bar for increments in the Faculty of Science has been slowly going up. As Dean, I view this as a good thing because it means we are moving in the right direction. Our most precious asset is our reputation and improved quality (and quantity) of research, teaching, and service will help our national and international standings. FEC is meant to be a positive process. Unfortunately, the number of requested increments is always greater than the number we are allowed to hand out. In 2005, there were 52 half increments that had to be clawed back by FEC – literally one in six faculty members received an unwelcome letter just before the Christmas break. In 2012, 27 people were disappointed to see that their Chair’s recommendation was decreased by one half of an increment. However, five people saw their incrementation go up by one half. The Chairs do a fabulous job of calibrating performance within their department. However, each Chair works in isolation of the recommendations of their counterparts in the other Science departments. It is only when FEC meets that the cross-department calibration can happen, and this is where the boundaries between increment levels gets set. If you lost a half increment, it is often because you were close to a boundary that fell on the wrong side of your Chair’s recommendation. Clearly these boundaries are subjective, but all the members of FEC work hard to make sure the process is fair and consistent.
Schaeffer
12
If you want to improve your chances at FEC, then there are some obvious words of advice Quality publication venues are important. One paper in a strong journal can be more important to FEC than several papers in mediocre venues. Use the “Additional Data” section of your annual report to help inform your Chair and FEC. For example, you can explain your choice of publication venues (e.g., maybe you chose a lower-ranked journal because it has a more appropriate audience for your work). Perhaps you want to provide more detail on your service commitments (such as the amount of work involved). Remember that a multi-disciplinary committee reads your annual report – help them understand what you accomplished in the past year. Work with your Chair to get advice on areas for improvement in your annual report. FEC values research, teaching and service. Little things can make a difference. For example, if you are getting low teaching evaluations, be proactive and get help. If your service load is low, take the initiative and volunteer to serve on a committee. Looking to the 2013 FEC, be warned that it will be more competitive than ever. It is not just because the Faculty of Science will be stronger with improved research, teaching and service. The reality is that there will be fewer increments available. The latest AASUA settlement makes available 1.175 increments per academic staff member, down from the current 1.20. That translates into roughly 7.5 fewer increments (15 half increments, or 30 quarter increments – quarter increments are new) to hand out across the Faculty. Getting extra incrementation in 2013 will be that much harder.
Schaeffer
13
I have a great job. I get to brag about the Faculty of Science. We need to congratulate ourselves on an excellent 2011-2012. Research quality and quantity were up, teaching is exceptionally strong, and service levels are impressive. This is reflected, in part, in the number of international, national, provincial, and university awards that our researchers are winning. As well, I am delighted to report that six people were awarded tenure, one FSO was given a continuing appointment, and 19 people were promoted (16 faculty members and three FSOs).
Schaeffer