PDA

View Full Version : Seeking ideas, comments, and criticism for untitled's aggregate results pages



untitled
10-29-2012, 09:14 AM
Hello all. Since things are getting pretty busy around here I wanted to remind everyone that I am super open to ideas, comments, and criticism of the aggregate roll call, admissions and rejections, and profiles and results work, here (http://tmanalysis.econwiki.com).

It was mentioned to me, for instance, that "the graphs for GRE scores for example only plot a few data points, and then there will be a huge list of profiles below that seem to have gotten missed."

This is because the script doesn't find a GREQ score and/or a GPA for that person or that the GREQ/GPA is out of range. Taking, for instance, the Yale 2012 results (http://tmanalysis.econwiki.com/schools/2012/yale.html), all of the accepted applicants I list are on the graph except lebatin. lebatin did not post on roll call or profiles or results, so I only have record of schools to which they posted acceptance or rejection. I still list them, because the correlation of accepted and rejected schools is useful and interesting even without the applicants profile. On the other hand, there are often small numbers of applicants who I don't find for other reasons. Please let me know if you find any examples, as I can often improve my code so that they are found.

Also I (obviously) need to improve my presentation so everything is more clear. Let me know if you have any ideas about that (or anything else) as well.

kipfilet
10-29-2012, 03:02 PM
You might be happy to know that a LOT of people who don't even know what TM or EJMR are use your results page as a reference.

untitled
10-29-2012, 04:47 PM
You might be happy to know that a LOT of people who don't even know what TM or EJMR are use your results page as a reference.

Hmm, definitely curious about that. Are there very many people applying to grad school who don't know what TM is? Are there very many american academic economists who don't know what EJMR is? Are there many people who aren't applying to grad school who look at my site? I suspect the answer is "no", "no", and "no", but you suggest that at least one of those is "yes".

Humanomics
10-29-2012, 11:04 PM
Again I think I speak for everyone with a huge "Thank You!"

Maybe, instead of listing the whole profile below the graphs, you could have the script clean them of the qualitative information, limiting to GPA, GRE scores, math courses, and research experience (dunno if everyone agrees those would be the best first-approximation variables to view). That way you could blast through a lot more candidates quickly to get a feel for ranges, and some sort of click-through link could take you to a full profile if one is interested in reading.

Ideally candidates would fill out a survey form with predetermined questions and answers, with an option to comment at the bottom. That might help standardize and organize everything.

untitled
10-30-2012, 11:38 AM
Maybe, instead of listing the whole profile below the graphs, you could have the script clean them of the qualitative information, limiting to GPA, GRE scores, math courses, and research experience (dunno if everyone agrees those would be the best first-approximation variables to view). That way you could blast through a lot more candidates quickly to get a feel for ranges, and some sort of click-through link could take you to a full profile if one is interested in reading.

Good idea. I could set it so clicking on the profile summary gives the whole thing (especially if I put in an "expand all profiles" button). Are those 4 factors the ones of interest for others?


Ideally candidates would fill out a survey form with predetermined questions and answers, with an option to comment at the bottom. That might help standardize and organize everything.

Yes, that would follow how they do it at lawschoolnumbers.com (http://www.lawschoolnumbers.com/) (my inspiration) and how they proposed doing it at thegradcafe (http://thegradcafe.com). I think Erin once proposed doing something like that here, but it never happened. If anyone wanted to make pages for that, I'm sure we could talk to buckykatt or Erin about hosting. In the meantime, there are advantages in the current system such as convenience (users can post roll call and admissions decisions separately and my script stitches them together based on username) and TM synergy (people are more likely to post their info because they want to be a part of my page and upon posting info are more likely to take part in discussions here at TM and vice-versa).

ssch
10-30-2012, 05:44 PM
Incredibly useful!!!

Thanks a lot!!!

Humanomics
10-30-2012, 07:48 PM
Oy. Gradcafe. I suggested a Roll Call thread in the sociology forum and got flamed about how it would only make people anxious and competitive, and that "Indeed, the perspective of numbers is very limited, but what would we expect from economists."

You make a good point, that there is sufficient network-externality-good momentum built up on TM that switching to a new standard format would probably be prohibitively costly. There's also the issue then of you having to integrate the new and old formats. Thinking on it -- data collection really isn't the issue here: we've got volumes, and nice consistent variables -- it's just the aggregation and presentation.

There certainly isn't a consensus on whether international applicants are comparable to domestic, but maybe for those who do not want to compare themselves to domestic applicants and vice verse, you could offer separate, and mixed views based on that too.

apo
11-02-2012, 02:53 AM
No please don't do that.

Information with whole profile is more useful.


Again I think I speak for everyone with a huge "Thank You!"

Maybe, instead of listing the whole profile below the graphs, you could have the script clean them of the qualitative information, limiting to GPA, GRE scores, math courses, and research experience (dunno if everyone agrees those would be the best first-approximation variables to view). That way you could blast through a lot more candidates quickly to get a feel for ranges, and some sort of click-through link could take you to a full profile if one is interested in reading.

Ideally candidates would fill out a survey form with predetermined questions and answers, with an option to comment at the bottom. That might help standardize and organize everything.

untitled
11-06-2012, 09:29 AM
Hmm, I think I'm getting mixed signals. What do you mean whole profile is more useful?

2013applicant
11-06-2012, 04:55 PM
I don't mean to take this thread in another direction, but I would like to see conditional probabilities implemented in some form or fashion. Take this for example: What is the probability of getting in to Harvard given that I have at least a 3.8 GPA? I think this could potentially add a lot to the analysis since these conditional probabilities could be added in for other measurables such as GRE scores, number of math courses taken, etc.

untitled
11-06-2012, 05:47 PM
Take the thread in any direction, that is great.

Certainly I could just count (9 out of 254 (3.5%) applicants to Harvard in my data with GPAs reported on a 4 point scale and over 3.8 were accepted). But I know this is wrong (I mention bias below). If you want to do your own analyses, I have a csv file with all my results, here (http://tmanalysis.econwiki.com/difficulty.csv). PM me if you want a codebook (you can probably guess the codebook, -1 is waitlist, 1 is admit, 0 is reject, the extra predictors are poor because it is hard to scan the profiles, but are not completely useless).

Also, I don't think I know exactly which conditional probabilities are of interest. I'm happy to answer any simple questions I can in the meantime. I really just need to write up a good codebook and clearly publish the data on the set and people can do there own conditional probabilities, but...

I'm hesitant to use my data to predict outcomes because it isn't hard to wait 6 months and find out your own outcomes, and my data is heavily biased. Posters at TM are highly above average (and probably above lurkers), so if I make a predict outcomes script, I will over estimate positive results for most applicants.

bholly
11-06-2012, 07:12 PM
I don't know how difficult it would be given the heterogeneity in the way the data is presented, but I think it would be great if you could distinguish more easily between funded admits, waitlist for funding, no funding, etc. That's something that would've helped me a lot in the last few days.
Even just make it so that, after each profile, where it lists the entries from the results thread, the school whose page you're on is listed first so that you can find the funding info more easily.

I'd prefer to have all information available, but wouldn't mind an option for just showing some basic data as long as there's still an 'expand all profiles' or equivalent like you suggest.

bholly
11-06-2012, 07:13 PM
I don't mean to take this thread in another direction, but I would like to see conditional probabilities implemented in some form or fashion. Take this for example: What is the probability of getting in to Harvard given that I have at least a 3.8 GPA? I think this could potentially add a lot to the analysis since these conditional probabilities could be added in for other measurables such as GRE scores, number of math courses taken, etc.

There's a little of that here:
http://www.urch.com/forums/phd-economics/127711-elo-ranking-schools.html

Although the data is likely a little outdated now.

chateauheart
11-08-2012, 08:02 AM
It would be nice just to sort the profiles in the list by US/international (for applicants to US programs). The profiles tend to be so differently structured that they're not really of interest to the applicants of the other type.

Having a separate page up for all the profiles would be even better. You can tag them with other categories besides US/international (e.g. applicants to top 10, applicants to top 50; applicants from low-ranked colleges, etc.) and have them sortable by each. This would be a lot more convenient for an applicant than going through all the profiles/results threads looking for a similar profile. Which as of now is probably the most valuable thing an applicant can do - so if you could improve this aspect, you'd definitely cause some welfare gain.