PDA

View Full Version : How does E-rater work?



minud
10-02-2004, 10:27 AM
How does E-rater work?

In early 2000 when the test-maker began to introduce e-rater, there was a great controversy on this computer program. People simply don't believe it. Some students even type "I don't want to be graded by a robot" in their responses on test day. Several years later, however, the e-rater proved to a reliable substitute for high-cost human grader.

The e-rater works like a search engine, such as the world-wide used Google. It is comprised of a computer program and a database. The database stores hundreds of sample responses to each essay topic. The computer program scans your essay and then determines its relevancy to the pre-written essays in database. If your essay is similar to the 6.0 essay, then your essay receives 6 point. If similar to 5.0 essays, then it receives 5.0 point and so on. In other word, if you write an essay that is very similar to a high-score essay in test-maker's database, then you will get a high score.

Fooling the e-rater is easy if you are able to identify what it favors and what it does not. In the following passage, we will introduce the most commonly used structures and words/phrases in 6.0 essays.

The E-rater favors transitional words

Ordinal numbers that introduce examples or reasons: first, second, third, first of all, etc.
Transitional words that relate each sentence to other: since, because, therefore, thus, etc.
Mood words that indicate the author's position: fail, ignore, overestimate, underestimate, exaggerate, misrepresent, overlook, etc.
Counter-evidence indicators: actually, despite, admittedly, except, even though, nonetheless, nevertheless, although, however, in spite of, do, does, may, might, etc.

minud
10-02-2004, 10:30 AM
So where can I read sample 6.0 essays to get idea as to how to write essays????

jacob
10-03-2004, 12:53 PM
Wow, I didn't know that the essay rating was done by the computer itself !
Well, that really necessitates an examination of sample 6.0 essays. I guess going through these essays would prove to be really beneficial.

The only problem is where can such essays be found. Anybody, any clues ? I suppose such essays are not freely available otherwise we would have all known about them by now.

Anyway informative article, have to keep in mind about those words.

serg14
10-03-2004, 02:09 PM
PowerPrep CD contains 3 issue and 3 argument essay responces graded by 6.0.

I'm not sure whether essays are scored by some software, but ETS says the following:

In general, GRE readers are college and university faculty experienced in teaching courses in which writing and critical thinking skills are important. All GRE readers have undergone careful training, passed stringent GRE qualifying tests, and demonstrated that they are able to maintain scoring accuracy.

- each response is scored by two readers
- the scoring procedure requires that each response receive identical or adjacent scores readers; any other score combination is adjudicated by a third GRE reader

Econ
10-03-2004, 03:56 PM
Actually I have submitted 8 essays to the the e-rater. 2 of them where in the practice mode and 3 times I wrote them on-line (1 argument and 1 issue each time). My opinion is that you can deceive e-rater easily. The first time it gave me a 5.5 (I5 and A6). The second time 4.5 (I4 and A5); the truth was that I was feeling very bored during my writting. Then I went through the comments the e-rater makes about your writting and I found out that to get a good score you have (i) to use transitional words (ii) to include a topic sentence in every paragraph (iii) that the e-rater is very sensitive to spelling and grammatical mistakes (contrary to the real GRE) and (iv) is not sensitive at all to the intuition of your writing and to the organization of your essay (e.g. the e-rater never identified my main point). Taking all these into consideration I took one more test and guess what.... 6/6 although my ideas where a little bid stupid, my examples where out of place and the e-rater did not identify any main idea in my essays. Just I had to take care to give a LENGTHY and free of mistakes essay.

In the real GRE, humans correct you so these tricks do not pass. I wrote an email to ETS about the defects of e-rater but I got no response. Anyway, the only good thing in this is that with only 10 $ you can take a test in actual conditions (because you think that this test will be scored fairly and thus you give the max test mentality).

sally82
10-03-2004, 06:00 PM
Hi Econ,
are you sure that the real GRE essays are read by only by human readers? I read on a web site that ETS changed that recently and now each essay is graded by e-rater and by one human reader. The scores of e-rater and the human reader are than average. Is correct? Did any of you guys hear about this change in ETS essay grading policy?


P.S.: This is a quote from a passage I found on www.800score.com

"The GRE used to use two human graders to grade your essays. If they disagreed, it went to a third grader. Under the new system, a human and the "E-rater" (a computer program that scan essays) will grade your essay. If the human and E-rater agree on a score, that's the grade your essay will receive. If they disagree, a second human will grade the essay to resolve any differences."

Econ
10-03-2004, 09:16 PM
I was pretty sure. I knew that e-rater was experimental, that humans corrected the essays and that software was used only to detect stolen material. Indeed 800com confirms what you say, but in the official ETS site, ETS says the opposite. Check it with: ''www.ets.org'' then 'get feedback on your analytical skills'', then ''faq'', then ''know how Scoreitnow! service differs from the actual GRE test'', and see the last two columns.

minud
10-05-2004, 02:28 AM
Econ is rt. I found this on gre.org -

"The scoring of the analytical writing section is the same whether the test is taken on computer or paper. Each essay receives a score from two trained readers, using a 6-point holistic scale. In holistic scoring, readers are trained to assign scores on the basis of the overall quality of an essay in response to the assigned task. If the two assigned scores differ by more than one point on the scale, the discrepancy is adjudicated by a third GRE reader. Otherwise, the scores from the two readings of an essay are averaged. The final scores on the two essays are then averaged and rounded up to the nearest half-point interval. A single score is reported for the analytical writing section"

I don't understand then why sites like 800score.com and other sites keep talking abt E-rater??

sally82
10-05-2004, 06:30 AM
Thanks econ and minud!

GREAWfailend
10-05-2010, 06:29 PM
I never believe it to be true and was deceived by other people over and over again who claim the non-existence of e-rater, but now look at what the ETS official website says
(GRE: For Institutions: How the Test Is Scored (http://www.ets.org/gre/institutions/scores/how/))
---------------------------------------------------------------------------


The essay score is assigned by a trained reader, using a 6-point holistic scale. The essay response is then reviewed by e-rater, a computerized program developed by ETS, which is being used to monitor the human reader.
If the e-rater evaluation and the human score agree, the human score is used as the final score. If they disagree by a certain amount, a second human score is obtained, and the final score is the average of the two human scores.

---------------------------------------------------------------------------

I always got 3.0 in AW (3 times) and always feel that I should write as Shakespeare to score 6.0, but I'm wrong!! The key is to know what criterion they use.