Placements, why they aren't the best method for school rankings

[XanthusARES]

So a lot of effort has been put forth on this site and in the minds of students to quantify the ranking of a schools. A lot of people with good intentions often mention looking at recent placements as a proctor for school quality, and as a way to consider the type of university you'll have a chance at after the fact. In one or two of my posts I may have even suggested this.

 

There's a problem with this, though. Here's what everyone assumes to be, if you control for outliers (i.e. someone coming from a no name school and getting placed at Stanford and vice versa), then there is a correlation between placements of previous students and potential future placements (i.e. if I come from school X and people have placed at schools similar to school Y, then I have a shot at getting placed at school Y).

 

The problem is that this simple regression doesn't take into account various covariates. As noble as the academic pursuit is, we all know that some people get a PhD in business so that they can work decent hours for a reasonable wage. Second location is a huge factor in location decisions. Our model assumes that subjects are going to select the best college they can be placed at regardless of location. Third our model doesn't account for the fact that not everyone wants to go to the best college they can. If someone wants to be at a more heavily teaching university, we shouldn't include their placement. Fourth, our time in the PhD program changes us. All of us. So we could enter wanting more than anything to be the next big name in our field, but maybe throughout the course of the program we find we really just want to live life comfortably.

 

I think you will find at most schools (perhaps only outside of the top 10), some placements are not strictly based on merit. So using placements as a proxy for ranking probably isn't your best bet.

 

I must say that everything we've said previously is true, though. So if you're coming from a top 25 school, I would expect you to get placed at a top 10 program. That would be an outlier. I would also say that it doesn't hurt to look at placements. If you see a grouping of placements at a certain tier this can definitely be a sign. Mostly what I'm trying to get across is that placements are definitely idiosyncratic. Don't feel alarmed if the last three placements were at relatively poor schools. Check to see why. Did they just not get any research out during the program? Did they do their undergrad, master's and PhD from schools in that state (which is a reasonable proxy for making the assumption they love that state)?

 

One caveat I will mention is for top 10 schools this doesn't necessarily hold. People from these programs can be uber competitive. They are more likely to be willing to move around the country to secure the best possible placement they can get. I'm not saying that people outside of the top 10 don't do this, but that there is larger variance in reasons for placements outside of the top 10 (i.e. if 4 people graduate from Wharton, 1 might select a placement based on other reasons, while if 4 people graduate from Oregon, 2-3 might choose based on other reasons).

 

As always this is just my opinion and your mileage may vary. I'd be interested in hearing other people's opinions.

[taxPhD]

In general, I agree with you, there are many many factors than influence placements beside just the quality of the PhD program. It's very tough to rank doctoral programs on one common metric.

 

Since the true quality is obviously not observable, placements are just a proxy for the true quality. And like any proxy, there is going to be noise. The question then becomes, do placements have more/less noise than other potential proxies for true quality.

 

This is where I kind of struggle to find a single proxy that performs at a reasonably good level. Thinking of some of the other things that rankings are based off of. For example, BYU ranks accounting PhD programs based on the research productivity of graduates 3 and 6 years post-graduation. However, this has a lot of things (maybe even more than initial placement does) influencing it as well, besides school quality. Or ranking based on the quality of the faculty publishing, but quality faculty don't necessarily produce quality doctoral students. I am not sure if placements perform any better or worse than these other proxies do.

 

Probably the best bet would be some sort of composite score (but even those have weaknesses, just read about the criticisms of rankings like US News). Some sort of average ranking of placement, faculty publication, and student post-grad publication.

 

Overall though, I think the best advice is to just not get too tied up in rankings as a means for selecting a PhD program. Do placements matter, yes. Does faculty publication quality matter, yes. Does research productivity of prior graduates matter, yes. They are all useful pieces of information to know for perspective applicants. But none of them are even close to a perfect measure of how program X will be for a particular student. As many of us know, finding a program that is a good fit research wise and culture wise makes a big difference. If a student is miserable and cannot get any mentoring on the research they are interested in, then odds are, the student will not graduate. Then it doesn't matter if the program places well, has good faculty, and successful alumni because for that student who did not complete his/her degree, none of that matters.

 

More particular with placements themselves, on flyouts, if placements come up with current doctoral students or faculty visits, don't be afraid to ask about offers versus placement if it seems like some people (on the surface) underplaced. I can think of examples at the school I graduated from of people taking jobs at say the third "best" school that they received an offer from. But the department head wasn't afraid to tell prospective students that that person also had offers from these other better schools.

 

Anyways...just like others, this is just my 2 cents.

Edited December 5, 2014 by taxPhD

[quanttastic]

In general, I agree with you, there are many many factors than influence placements beside just the quality of the PhD program. It's very tough to rank doctoral programs on one common metric.

 

Since the true quality is obviously not observable, placements it just a proxy for the true quality. And like any proxy, there is going to be noise. The question then becomes, do placements have more/less noise than other potential proxies for true quality.

 

This is where I kind of struggle to find a single proxy that performs at a reasonably good level. Thinking of some of the other things that rankings are based off of. For example, BYU ranks accounting PhD programs based on the research productivity of graduates 3 and 6 years post-graduation. However, this has a lot of things (maybe even more than initial placement does) influencing it as well, besides school quality. Or ranking based on the quality of the faculty publishing, but quality faculty don't necessarily produce quality doctoral students. I am not sure if placements perform any better or worse than these other proxies do.

 

Probably the best bet would be some sort of composite score (but even those have weaknesses, just read about the criticisms of rankings like US News). Some sort of average ranking of placement, faculty publication, and student post-grad publication.

 

Overall though, I think the best advice is to just not get too tied up in rankings as a means for selecting a PhD program. Do placements matter, yes. Does faculty publication quality matter, yes. Does research productivity of prior graduates matter, yes. They are all useful pieces of information to know for perspective applicants. But none of them are even close to a perfect measure of how program X will be for a particular student. As many of us know, finding a program that is a good fit research wise and culture wise makes a big difference. If a student is miserable and cannot get any mentoring on the research they are interested in, then odds are, the student will not graduate. Then it doesn't matter if the program places well, has good faculty, and successful alumni because for that student who did not complete his/her degree, none of that matters.

 

More particular with placements themselves, on flyouts, if placements come up with current doctoral students or faculty visits, don't be afraid to ask about offers versus placement if it seems like some people (on the surface) underplaced. I can think of examples of the school I graduated from of people taking jobs at say the third "best" school that they received an offer from. But the department head wasn't afraid to tell prospective students that that person also have offers from these other better schools.

 

Anyways...just like others, this is just my 2 cents.

 

Once again TaxPhD is providing expert advice.

 

I would like to add just a quick snippet about placements. Placements are very individual specific and influenced by a ton of factors (spouses, geography etc). I would argue that some places with "varied" placements are actually really great places. The variety in placements means the faculty allows students to "follow their heart", and doesn't provide too much pressure based on rankings.

 

However one thing you might do is to look in the past decade at a program and see where the "best" 2-3 placements are. All else equal, this will give you an idea of where you could potentially place if you did great in the program. If the 2-3 "best" placements are ranked below the type of place you would like to end up, it is a bad idea to go to that program.

[XanthusARES]

However one thing you might do is to look in the past decade at a program and see where the "best" 2-3 placements are. All else equal, this will give you an idea of where you could potentially place if you did great in the program. If the 2-3 "best" placements are ranked below the type of place you would like to end up, it is a bad idea to go to that program.

 

This is phenomenal advice.

[PhDPlease]

 

However one thing you might do is to look in the past decade at a program and see where the "best" 2-3 placements are. All else equal, this will give you an idea of where you could potentially place if you did great in the program. If the 2-3 "best" placements are ranked below the type of place you would like to end up, it is a bad idea to go to that program.

 

Maybe my confidence is too low but I'd personally put a lot more emphasis on the median placement. I don't really have any reason to believe I'd be one of the most exceptional people in the program. I agree w/not going if even the best placements are below what you'd be happy with, but I'd personally want to also be reasonably happy enough with the median placements a well. But I guess this depends on your tolerance of risk/whether you think you'd be above average/etc.

[wittmic]

I have heard schools speak with regards to placement in the following way on multiple occasions:

 

"We can get you in the room with just about any university you'd like. The rest is up to you."

 

The point is, the program, the faculty and your adviser can all get you so far. The rest is up to you, how good your paper is and how well you present it, your ideas and yourself. Thus I think using the top few placements as a gauge on what's possible is a good tool. It basically says, program X can get you in the room with Z and Y. The rest is on you.

[isman]

Another important factor is the faculty connections. Some professors have great rapport with certain schools and if they move to other programs, it'll definitely impact the placements of graduates.

[simplypoet]

But how do you rate faculty connections, like there is a Prof. in a School I have applied who has awesome connections.

But looking at his/her research work he/she has never published with grad students.

 

But should we prefer ?

 

A Prof who has good connections or a Prof who publishes with grad students ?

[PhDPlease]

I would maybe see if the one who works w/grad students could be your primary adviser or you could RA for him if your school has you RA during the 1st or 2nd year, and then have the one with connections as a member of your committee not your adviser (assuming the one who doesn't publish w/grad students is willing to at least occasionally provide some advice). It also depends *why* this person hasn't published with grad students. Sometimes the professor has no interest in grad students, but other times there just haven't been any students who had interests that were a good fit w/the professor at the school in recent years.

 

Also I think this depends on the rank of the school. At a top-tier program, some professors are more famous than others, but mostly all tenure-track profs would be reasonably well-regarded even if they are not "famous". If you go further down in rank, you might have more of an issue where not all professors have a research reputation.

[XanthusARES]

But how do you rate faculty connections, like there is a Prof. in a School I have applied who has awesome connections.

But looking at his/her research work he/she has never published with grad students.

 

But should we prefer ?

 

A Prof who has good connections or a Prof who publishes with grad students ?

 

 

If a professor hasn't historically worked with students, then chances are they won't work with you. Sorry. There is always the chance that like PhDPlease has said you can maybe try to work with them, but my experience is that professors who don't work with grad students, don't work with grad students.

[Sheetrockero]

All great advice and insight... but how have you guys amassed such knowledge about programs without actually being part of the "in group"? I feel like a lot of this is insider knowledge that you become privileged to only after being immersed in academia professionally for a few years. Man, I wish I would have been more proactive about pursuing this career path while I was at least still in the same state where I did my MBA so I could more easily reconnect with Professors... Ok, enough woulda, shoulda, coulda. I have three more apps to finish by tomorrow.

[rsaylors]

All great advice and insight... but how have you guys amassed such knowledge about programs without actually being part of the "in group"? I feel like a lot of this is insider knowledge that you become privileged to only after being immersed in academia professionally for a few years. Man, I wish I would have been more proactive about pursuing this career path while I was at least still in the same state where I did my MBA so I could more easily reconnect with Professors... Ok, enough woulda, shoulda, coulda. I have three more apps to finish by tomorrow.

It is very hard to get recommendations or to understand where you're going, what you're doing, and what your new role is without some great mentorship and guidance.

[PhDPlease]

Try not to worry about it too much at this point. Personally I hadn't seen 2 of my recommenders in a few years before applying. Visiting them in person is a good idea but I am sure many people either don't think of the idea or live too far away for it to be feasible in terms of time/$$ to travel. I think on this board there is more of a tendency for people with good ideas to post and less of a tendency for people who did not do things to post, or for people to post "retroactive" analyses that are more coherent than their actual thoughts were at the time, which can make it seem like everyone except for you is an expert which it isn't actually the case.