Thursday, January 12, 2017

Recommendation Letters – How Students are Evaluated

At the end of last semester, from about mid-November to mid-December, it seemed to me that I was writing more student recommendation/reference letters than usual — which is fine, of course — but, as I was doing so, it occurred to me (as it does every year) that students requesting such letters fall into different categories. There are students who are good at everything, students who are good at some things, pretty good at others (e.g., good at composition, okay in theory), students who are pretty good at 1-2 things, rather weak in others, students are are really good at one thing (e.g., composition) but really weak at others (e.g., theory), and every permutation of these and other variables you can imagine.

If I am asked to write a letter, the academic areas on which I comment are the ones that I teach, namely music theory  and composition. But, as you can see in the list below, professors are often asked to rate students in a surprisingly-large number of areas that go far beyond specific academic disciplines, and so I thought it might be useful to list these areas, and, more generally, provide information on the evaluation process for anyone that may wish to request such a letter at any point in their future.

If you are in a Bachelor of Music programme and there is even a remote chance that you might apply to graduate school upon completing your degree, today's post may have some useful information for you. And, if you have no interest in this topic, I promise to get back to composition-related topics in my next post!

Asking for a reference letter
 • Don’t be shy about asking a professor to write reference letters; it is a part of our job, like correcting/grading, office hours, extra help for students that seek it, committee work, research/composition/performing, and of course, teaching.

• Be aware, however, that it takes time – for me, typically about 1-2 hours – to write a thorough reference letter.  

Why does it take so long?
Part of that time is spent going through a student’s transcript carefully (which I request from the student), reviewing how the student did in my courses (which involves opening the spreadsheet files for every course taken with me in order to see the student’s results in all parts of the course evaluation scheme), and ranking that student’s final mark in relation to the class (e.g., 3rd out of 34 students), because some graduate schools ask for that information, and if they don’t, I usually provide it anyway.

Time is also spent listing all of the positive attributes I can think of for that student, particularly those that I believe would be relevant to studying composition or theory at the graduate level.

I also list any weaknesses or concerns I have about the student, because admissions committees want a balanced and honest assessment; if I describe a student in only positive terms, but the transcript reveals that the student has a 75% average over the most recent 1-2 years, then it seems unlikely that an admissions committee would take my letter seriously; a B+ average is nothing to be ashamed of, but it’s not an A average, and, in my opinion, a professor's recommendation letter should expand on the evidence found in a student's transcript, without making claims that are unsubstantiated in the transcript.

What Student Attributes are Evaluated?
Professors are often asked to rate, and comment on, specific personal attributes or character traits, such as:
  • Ability – All around
  • Ability to work under pressure
  • Ability to work independently/Willingness to take responsibility for learning
  • Academic potential
  • Academic preparation/Demonstrated academic ability
  • Achievement
  • Communication skills – Oral
  • Communication skills – Written
  • Curiosity – Willingness to explore new approaches, ideas, particularly those that may fall outside the student’s comfort zone
  • Industriousness
  • Initiative
  • Intellectual capacity
  • Leadership
  • Likelihood that my institution would accept the student, if we had a comparable programme
  • Likelihood that the student will complete the degree
  • Organization - Planning
  • Originality/Willingness to “think outside the box”
  • Creativity/Talent
  • Persistence 
  • Punctuality
  • Research ability
  • Response to constructive criticism
  • Teaching ability (if known) 
  • Teamwork (work well in a group)
The rating system varies, but it often consists of a series of clickable boxes that correspond to percentile values, such as top 5%, top 10%, top 20%, top 50%, lower than top 50%, and "no basis for judgment." The professor clicks on the appropriate box, in their assessment, for each category.

Universities typically ask for a letter, in addition to asking the professor to rate the student in a number of categories, as above, but no university with which I am familiar asks for the student to be assessed in every  category listed above; the above list is compiled from a variety of different universities' lists. Also, some universities have comment boxes in their on-line recommendation forms that allow professors to add a sentence or two explaining our rating in that particular category.

So What?
I have no idea how much weight is placed on this aspect of the evaluation process – I suspect that it matters, but perhaps not as much as the student’s transcript, composition portfolio, and the recommendation letter, but that’s just my guess. At the very least, however, these categories tell us something about what graduate schools value in prospective students.

How would you assess your level in each of the above categories? Most people have at least a few weak areas; if you can identify your own, I encourage you to work at them so that they become strengths.

How to request a letter
When requesting a letter from a professor, give them lots of lead time, and provide all necessary information. More specifically:
  • Make your request at least two weeks in advance, because of the time it takes to write these letters. If this is not possible for some reason, then at least one week in advance, but be aware that professors are generally pretty maxed-out in terms of demands on their time, and if you don’t give us enough time, we may not be able to write the letter you request.
  • Make your request in writing, as well as in person (if feasible). The reason for this is that I don’t necessarily remember every request, but if you put it in an E-mail, then I have something to remind me. Confirming this request in person is mostly a social nicety, but it may also give you a sense of the degree to which the professor is willing to write you a positive letter.
  • If you are asked to include a composition portfolio, ask your composition teachers for their opinions on what to include, and make as many of the changes/improvements suggested by your instructor as possible. If applying for a theory master's and they require the submission of a theory essay, ask your theory teachers for suggestions and corrections on any essay you plan to submit.
  • Provide the professor with an up-to-date transcript if you can, for reasons described above. The transcript can be a series of screen shots taken on your computer from your “university self-service” portal, or a PDF of the actual transcript.
  • Provide a list of every course you took with the professor, and the semester in which you took that course.
  • List any relevant achievements that might strengthen your case, e.g., Senior Rose Bowl Winner, Kiwanis Festival of (what region or city), Gower Band Terra Nova Competition Winner, etc., with the date(s) of the awards. You could include non-musical achievements as well, e.g., cycled from St. John’s, NL, to Victoria, BC, April-August, 2014... If I think they reflect  strengths in your character, I might mention them. And yes, one of our students did this!
  • Indicate in writing every school for which you’d like a letter.
  • Include the submission deadlines for each school.
  • Indicate in writing the specific graduate programmes to which you are applying (e.g., MA in music theory, MMus in composition, etc.).
Electronic Submission?
Most schools allow professors to submit their letters electronically; I much prefer this, so if that’s an option, please choose it. I would guess most professors find E-submissions most convenient, but double check with the professor to see if they prefer to submit the old-fashioned way (i.e., a hard copy via "snail mail").

On the pros and cons of “Gentle Reminders”
To the best of my knowledge, universities automatically send applicants an electronic confirmation message every time they receive a recommendation letter for that student (UPDATE: Maybe it's a glitch in their systems, but they occasionally don't, according to a student who just contacted me… However, I think they are all supposed to do this). If the deadline is near, and you have not received such a notice, here’s what I suggest:

Send the professor a “friendly reminder,” or “gentle nudge” a few days before the submission deadline . For professors, work life tends to be pretty hectic all semester, but particularly so towards the end of a semester. Also, if you have, say, eight students who each want three reference letters, that’s twenty-four references; that's a lot of references, and it is possible to miss some. Not only that, but E-mail is a somewhat inefficient way to communicate, in that every day our inboxes are bombarded with SPAM and “noise” (messages that are not SPAM, but clutter up our in-boxes), so it is easy for the occasional legit E-mail (one that requires a response) to get buried among the debris.

I know that not everyone appreciates reminders, however – I remember as an undergraduate student seeing a professor in the hallway one day, and reminding him as politely as I could that the deadline for that letter was that day, only to have the professor blow up at me and proclaim, in an outdoors (not indoors) voice, “Look! If I say I’ll write you a reference letter, I’ll write you that g**d*** letter!”

It occurred to me that I might have destroyed any hope I had of going to graduate school, which I found rather daunting at the time. In today’s world, however, this kind of situation can be mostly avoided, because a thoughtfully-worded gentle reminder via E-mail is unlikely to elicit such a rude response. Speaking only for myself, I don’t mind them at all.

Trust the Process; How do you know the professor will write a good letter?
Speaking of daunting, it can feel a bit scary to request a letter and then trust that the professor has written a good letter. But, once your request has been made and the professor has agreed to recommend you, I would suggest trusting the process. In my opinion, a conscientious professor should tell you if they can’t write you a good letter, rather than agree to write a letter, and then write only negative things about you.

I have turned down requests to write recommendation letters on very few occasions — when I do, I always suggest that the students find other professors to write recommendation letters —  for one of three reasons:

(i) The student's request came too close to the submission deadline. This is just a practical matter, not personal — requests often come at the busiest time of a semester, and there are times when I cannot set aside all the other work I have to spend two hours writing a letter that is due the day after I got the request. On the other hand, if I have recently written a letter for you, and you ask me to send it to more universities a day or two before a deadline, this is less problematic. It still can take a little more time than you might expect, particularly when the original letter needs to be modified in any way (such as if the original letter needs to be updated due to new information), but if I can do it, I will. But I'd still prefer 1-2 weeks notice, of course;

(ii) I don’t know the student well enough to say anything really positive about her/him; e.g., they only took one course with me, and it was a few years ago, and the student did not get a very good result in that course;

(iii) I know the student pretty well, but what I know isn't particularly positive… Perhaps they took several courses with me, but did not do very well in any of them, or they were rude/disrespectful on multiple occasions, or they skipped or were late for classes on multiple occasions, or they missed assignments or quizzes several times, or they committed academic dishonesty (the most common example of this is students collaborating with each other on assignments), or they generally disregarded my compositional advice and suggestions, etc.

To be clear, I’m not talking about an occasional disagreement with a student; these are not unusual in teaching, and I don't mind them, as long as the discussion is respectful. I’m referring more to an on-going and troublesome pattern of behaviour, which is extraordinarily rare; I can think of only about three such cases in thirty years of teaching.

The only time I recall declining a recommendation letter request for reason (iii) above involved a student who was extraordinarily sullen and rude. Don't get me wrong — I have encountered occasional instances of rudeness over the years, but this particular student really stood out from the rest (and not in a good way!), so much so that I actually had other students complain to me about the student’s behaviour. I was therefore surprised when the student asked for a recommendation — I'm not sure what sort of karmic good-will credits they thought they were accumulating while engaging in disrespectful behaviour — and so I suggested they would be better off asking someone else for a recommendation.

I don't hold grudges, or try to undermine students in any way. Had I decided to write a letter for this person, I would have found positive things to say about her/him (no matter who you are, I can usually find positive things to say about you, and this student was definitely smart and talented), but I would have also had to mention the problematic behaviours, which I felt would have sabotaged their chances of getting into a graduate programme, which is why I suggested they ask someone else.

Not sure about applying?
If you want to go to graduate school but are thinking that perhaps you ought not to apply because your marks aren't great, or your composition portfolio is not strong enough, I would suggest applying anyway, because you never know what the outcome will be. Sometimes, average students (IMO) have been accepted into master's programmes, while stronger students have been rejected. You don't know the circumstance or criteria used in the evaluation of your application; in some years, and in in some universities, you might be in competition with sixty other students; other years, you might only be in competition with eight other students for six available spots.

If graduate schools reject you, and are not willing to give up on your dream, find a way to turn this into a positive experience: Identify your weaknesses, work hard on improving in those areas, and then apply again. For composition, this might entail writing more pieces, perhaps longer and more substantial, perhaps for larger ensembles, or written more idiomatically for the instruments involved, or exploring new techniques, or providing more score detail, or better recordings, or just writing better pieces; most of us acknowledge that our early compositions are not as good as later compositions. If you had some weak results in music theory classes, explore the possibility of taking some of them again, this time working harder than you did previously, in order to gain a better understanding of the material.

Types of applicants
It is easiest students to write strongly-positive recommendations for students with these qualities:
  1. At or near the top of every composition course they took with me;
  2. Developing into very good composers — this is slightly different than #1, because a students course grade can be near the top of the composition class by getting full marks in all the extra-compositional assessment areas — excellent preparation, excellent contributions to class discussions,  making the required number of thoughtful composition blog comments — while producing compositions that are pretty good (B+, A-), but not extremely good (A, A+);
  3. At or near the top of every theory course they took with me;
  4. Actively seek out new scores to study and new music to hear, and are excited by new discoveries;
  5. Willing to try new things without pre-judging them, and to embrace at least some of them;
  6. Do all their work on time with a minimum of excuses;
  7. Do more work than most other students;
  8. Respond well to compositional suggestions,
  9. Have a great attitude, and
  10. Participate regularly in class, and, in particular, they are good at giving constructive criticism to fellow students in our composition seminars.
If I were assigning a score of 1 (= terrible!) to 10 (= whatever is better than excellent!) to each of the above qualities, I feel confident in saying that I have never had a student who I would have rated as a 10 in every category.

However, I have had occasional composition students who are strong in all of these areas, meaning I would perhaps rate them as an 8 or higher in every category. Such students tend to be accepted into graduate programmes, provided they didn't only apply to the most selective universities, like Indiana or Yale (but our students have gone to Indiana University and Yale, for example, so why not you?).

What do you do if you are pretty good at composition, but pretty weak in, say, theory?
 Most students are strong in some of the areas, but not all, and in many cases such students have been accepted into graduate programmes. Some students are very good at composition, but very weak in music theory; this can be a deal-breaker for some universities, since a graduate programme in composition usually involves music theory courses as well. I still recommend that such students apply to graduate schools, and, if not accepted, then consider doing extra work in music theory (like re-taking music theory courses to try to obtain better results), and then applying again in the following year. Realistically, however, this does not give students much of an opportunity to improve their standing in theory courses, since applications are often due by the end of the fall semester, which would give a graduating student only one semester in which to demonstrate an improvement.

If you are reading this in the early years of your academic studies, the clear conclusion to draw from this is to become as strong as possible in every possible area. It starts with taking every course seriously, and not falling into the trap of making excuses for poor results in, say, theory courses, such as, "this is nothing but a set of rules for OLD music; I want to make NEW music, and break any rules I wish!"

I will confess to speaking from experience on this matter. When I started taking music theory, I could not relate to it, and thought it had nothing to do with the music I wanted to write. Consequently, I failed grades 3 and 5 harmony at the Royal Conservatory of Music (RCM), and didn't do very well in grade 4 harmony either. After about two years of poor results at the RCM, I started studying privately with an excellent composition teacher, Dr. Sam Dolin, who made me redo music theory from the beginning all over again, and, while I resented it a bit at first, he made it interesting, fascinating, and relevant for me, and my love for music theory – which is, basically, learning how music works – has never abated. And yes, I got better at it.

You can get off to a bad start in something, and end up becoming very good at that thing; in my case I just needed a combination of a good teacher and an attitude readjustment.

This has become much longer than I anticipated, but if you have questions on anything relating to recommendation letters that I did not address in this post, please ask them in the "comments" section, and I'll try to respond.


Flutiano said...

As someone who is waiting to hear back from Master's programs, and hopes to one day be a professor who will write letters for students, it is interesting to read this perspective on recommendation letters.

I find the process that you (Dr. Ross) describe as preparation for writing a recommendation letter intriguing. Are you looking for particular things when you look at the transcript? Do you ever share with students where they have ranked in the classes that you took with you? Does looking at the spreadsheets remind you of things that they did well or poorly on in your courses that you would then include in the lists of positive attributes and weaknesses/concerns?

The deciding to and where to apply for master's programs is an interesting dilemma. Now that I am waiting to hear back from two schools, and the school that I did hear back from was a rejection, I am wishing that I had applied to more programs. Part of why I didn't apply for more composition programs is that I do feel like my composition portfolio is lacking, as are my compositional skills. It is hard to take rejection as a positive experience, although my anxiety about not getting in anywhere has led me to making plans for next year to the end of improving my portfolio and my skills and reapplying for the following year.

Clark Ross said...

Quick answers to Flutiano's questions:

Are you looking for particular things when you look at the transcript? Basically, I'm interested in (a) trends (are they steadily improving, are they really strong in one area (e.g., composition), but really weak in others (e.g., theory)), (b) how they did in theory and composition courses I didn't teach, (c) how they did in ear-training and dictation courses (doing well in these courses is, IMO, a very significant indicator of a student's potential to do well in any aspect of music, although there have been rare cases where a student struggled in these areas and still did well in other aspects of music), and, more generally, (d) how they did in all other courses.

Do you ever share with students where they have ranked in the classes that you took with you? I am willing to do so if asked, but I don't volunteer this information.

Does looking at the spreadsheets remind you of things that they did well or poorly on in your courses that you would then include in the lists of positive attributes and weaknesses/concerns? Some students do poorly in part-writing, but well in analysis, and vice versa, and looking at my course spreadsheets can remind me of this. Spreadsheet summaries can also reveal that some students do well in assignments, less well in tests and exams, which can be a concern, depending on the degree of discrepancy between the two. Sometimes the spreadsheets show that the student is really good at everything, which is the ideal situation, because it is a pleasure to point something like this out in a reference letter.

Robert Humber said...

Waiting to hear back is an anxious time.

I think it is really great that you pay attention to detail in your reference letters. I didn't really know what they entailed until now, and had no idea there was a survey-type thing. I obviously have no proof of this, but I have suspicions (and have heard) that some professors will write cookie-cutter letters. This doesn't necessarily hurt, but it also probably doesn't help... kind of just a nothing letter. By going into detail, you show an understanding of the student and give the committee something to consider. It seemed to work with UBC! :)

Alison Petten said...

Reading this post as a second-year student is both relieving and nerve-racking. Previously I had the mentality that all that was necessary for graduate school was a good transcript and developed musical skills, but after reading I now realize that students need much more than this to be successful in graduate applications. It is also very comforting to know that there are professors at our university who put so much thought and detail into reference - it's great to see faculty that care as much about their students as ours does!

Peter Cho said...

I'd like to add a few points regarding the anxiety that may surround asking for a letter of recommendation from a professor. The question of whether the letter will be positive or negative is certainly a worry. However, I found that in the process of applying to grad schools this year that there was an altogether different source of fear. I found that I a fair amount of anxiety stemmed from an almost complete unfamiliarity of the process of applying to grad schools. I trusted in the schools to make a fair judgement of my application. I also had complete faith in my referees to give an honest opinion of me in their letters of recommendation. But I still felt that I was doing this for the first time and had very little idea of the inner workings of the whole process. The most nerve-wracking part of this is that you get one shot at applying for grad schools (sure, you can re-apply the next year but that means you are left with a year with nothing planned to do). I wish that as a part of our undergraduate education we would be more informed on the processes of evaluation for grad school applications (and, as an extension, job applications and PHD applications). Even reading this blog post informed me on things regarding grad school applications that I had no idea about. If I had known these things a few months ago, it would have eased some of the anxiety I had about applying to grad schools. Knowing as much as possible about the evaluation process of grad schools would probably allow students to create stronger applications, or at the very least it would ease some of the fear of applying to grad schools.

Erika Penney said...

After reading this blog, it has given me a greater understanding on how important reference letters are. I have been told multiple times that not only are transcript grades and success in your degree important, but your references are also a big contribution. It does give students a bit of anxiety because they are not completely sure what their professor may say about them. Knowing that a teacher is completely honest while writing is different for some students because they may have good grades and great work, but maybe not the best punctuality or attitude, etc. It is great to know that professors take the time to write these letters because they want to see their students succeed in their future studies.

Benjamin Taylor said...

It is very interesting finally getting an insight of how you (and maybe some other professors) write reference letters. While finishing up my applications to Master Schools, I assumed you wrote a letter of some sort and sent it to them via a link the schools provide but it is interesting to hear that it is in a survey format. Even though many people think a GPA is everything, it is nice to see a point of view from the professors' side when it comes to the hidden side of applications and that reference letters (which now make sense) are close to almost as important as a GPA, portfolio or audition tape.

Julianne Meaney said...

These past few months I have been in the process of applying to Summer programs, and the reference letter section is one of the portions I feel the most stress about. Unlike the audition video or my resume, I have no control over this section. It is really helpful to know what criteria you use to write your letters, which I'm sure is quite similar to the ones my professors use when writing mine. There truly is so much more to a person's suitability for a program besides their GPA, CV and audition tape, and it's nice that these letters can help communicate these factors as well as what the rest of the application shows.

I also wanted to comment on your anecdote about your personal struggles with music theory. I started studying music theory later than most of my peers, and it did hold me back near the beginning of my degree, partially because I believed that I wasn't naturally good at theory and therefore would never be good at it. A shift in attitude certainly helped me improve my grades and my feelings towards the subject. It's certainly very encouraging to hear how far you have come, so thank you for sharing!