TestMagic Forums - View Single Post - Meeting with NABP: Are you ready to talk to NABP?

knok · 04-10-2008, 05:20 AM

TOEFL iBT Complaint:

Subject: TOEFL iBT Complaint

June 19, 2008

Dear_____

We are a group of the Test of English as Foreign Language (TOEFL iBT) test takers who are concerned with the test practices of Educational Testing Service (ETS). We would like to express our frustration about the testing center availability, testing environment and equipment, and test scoring process. ETS has published many reports and official guide to give an impression that the TOEFL iBT is given fairly. However, the speaking test is too subjective and the evaluation is too vague for test takers. Hundreds of test takers continue to have bad experiences with TOEFL iBT test and services while they have paid a high testing fee of $150 for each test. We would like your office to help us with this situation.

TOEFL iBT Complaint

1. Defective and Low-Standard Equipment

Overall, Thomson Prometric testing centers are bad, others are worse.

1.1 Computer Monitor

Some Thomson Prometric testing centers are using old monitors, which are not the LCD type. Some computers used in this test center should be thrown away since they do not provide good images anymore. They are hard to read because they blink. Some computers are not set to have high contrast between images and background. This makes it hard for test takers to read the passages or, if they can read them, they will have to stare at the screen intensely and hurt their eyes. Particularly, test takers are required to read perhaps five passages on the reading section, for a total time of two hours for the reading section alone.
This problem usually occurs at Thomson Prometric testing centers. When test takers asked to change their monitors, the test proctors responded that this was the way they were set up and they had nothing to do with them and suggested test takers to write the complaint to ETS directly. Unfortunately, nothing had been resolved after several hundred test takers filed their complaints to ETS. This problem has continued for years.

We suggest that LCD monitors, which offer the latest technology be used for the current TOEFL iBT. This type of monitor can be acquired at less than $170 each.

1.2 Internet Connection Errors or Computer Problems

In addition, in some test centers, it takes too long to see another question after clicking the answer on the previous question. It is not fair for test takers because this test is giving in a time constraint. On some occasions, test takers cannot see some buttons on the test or they cannot change the answers after clicking on the answers.

It may be true that test takers can raise their hands and request a test proctor to stop the test and re-operate it but in some occasions it is impossible because test proctors do not see test takers when they need help immediately. Test takers waste their test time at least 5-10 minutes asking for help. Also, in the reading section, the information that test takers already read before calling test proctors is lost from their memory. They must re-read it again. This results in a lower score on that section. This problem usually happens at some other testing centers that ETS uses, such as high schools or colleges. These facilities have limited hardware or Internet capacity. Most importantly, they set tables too close to each other with no partition, less than one meter apart.

1.3 Insufficient Paper Notes and Pencils

Furthermore, test takers must take notes during the test. Some test centers give only a few pieces of paper to test takers and tell them to ask for paper later when candidates use it up. When candidates run out of paper during the test, it is almost impossible for them to ask for paper in the middle of the test. This is because it is not convenient and time consuming to ask for paper. Also, some test centers do not provide enough sharp pencils to candidates. Writing with a dull pencil will reduce the speed of taking notes which reduces ability of test takers to perform well, especially on the Speaking and Writing Section.

2. Disturbing Testing Environment

2.1 Noisy Environment

Most importantly, the test environment should be quiet at all time. Nobody should be allowed to speak in a test room at any time. First, when people are testing microphones at the beginning, some test takers are working on the reading section. It is difficult to focus and perform well in answering questions. Second, when some test takers are listening to the passages on the listening section, others are shouting to answer the six speaking questions. Remember that test takers are checked in at different times so the time of speaking may take longer than 40 minutes. It is hard to concentrate on listening or writing when they must be in distracting environment.
2.2 Unprofessional Test Administration

Moreover, the test proctors, who should be ensuring that we have a proper environment for taking the test, are sometimes the biggest problem. They have been known to carry on conversations while the test is in progress. Some test proctors even sign candidates in, take pictures of candidates inside the test room, surf the Internet next to test takers while the test is in session, and even listen to music without headphones.

3. Format and Time Constraint in the Speaking Section

ETS states:

The raters who listen to your response will analyze it in three general categories. These categories are Delivery, Language Use, and Topic Development. All three categories have equal importance. Use the sample Independent and Integrated Speaking Rubrics on pages 242 to 246 to see how responses are scored. This section includes important points that should be covered when answering each question. All of these points must be presented in a response in order for it to receive the highest score in the Topic Development category. These important points are guides to the kind of information raters expect to hear in a high-level response. (ETS official guide to the new TOEFL iBT, p.233)

ETS also published the strategies for raising the TOEFL Speaking score by telling test takers to ask themselves these questions:
Did I complete the tasks? Did I speak clearly?
Did I organize my ideas clearly and appropriately?
Did I provide a complete response?
Did I use the time effectively?” (ETS official guide, p. 227)

The ETS official guide recommends candidates to time themselves when practicing the speaking tasks so that they can get an idea of how much can be said in the allotted time (ETS official guide, p. 247).

Officially, in the Speaking Section, there are only six questions. The first two questions are called Independent Speaking Tasks because they require test takers to draw entirely on test takers’ ideas, opinions, and experiences when responding. Test takers have 15 seconds to think and 45 seconds to speak. It is really difficult for test takers to think and formulate their thoughts in 15 seconds and organize and speak in 45 seconds, particularly when the questions are not familiar in their daily lives.

For the rest of the questions, candidates are required to summarize the conversations, lectures, and reading provided. In some tests, the topics are too difficult or are not in the interest of test takers. This test is unrealistic for measuring speaking ability of test takers. It is the test that puts more focus on organizing speech in the time limit. ETS mentions on its ETS official guide that test takers may not finish the tasks but can still get a high score and, in the meantime, those who can speak fluently may not get a high score in the Speaking section test if they cannot answer intelligibly and sustainably.

Not surprisingly, after asking many test takers, even though they can speak fluently, most of them are very nervous during the test and cannot finish answering the questions, none of them can get a score higher than 24. ETS informs institutions that a score of 26 is a score that is equal to a score of 50 on the Test of Spoken English (ETS official guide, p. 350). However, a score of 26 on the speaking section of TOEFL iBT is currently a minimum score for professionals to fulfill their requirements for their professional licenses.

According to TOEFL® Test and Score Data Summary for TOEFL Internet-Based Test: September 2005-December 2006 Test Data (ETS, page 9-10), the mean score of the candidates from the United States on the Speaking section of TOEFL iBT was only 22 whereas the mean score of the candidates from countries in which English is their primary language is 23. Even though some of these Americans may not have grown up speaking English in America, it is important to recognize the relatively low score for those who have been in an English speaking country for their entire lives.

This result indicates that the test is invalid and unrealistic in the real world. This test seems to measure intelligibility or other aspects rather than the language ability. This seems to be unfair for test takers when it is obvious that getting a high score on the Speaking section has nothing to do with the ability to speak English fluently.

4. Grading System:

4.1 Transparency of Recording

In TOEFL iBT, in the speaking part, a narrator tells candidates to check the microphone and one warm-up question is given to check its system, but how can we know that the volume and clarity of our voice is sufficient to be understood or clear enough to produce a good impression on the raters? Test takers will never know whether their recording is done properly or not because we cannot hear our own response.

Obviously, ETS has candidates sign a document before taking the test stating that after finishing the test, all exam results or recordings will solely become a property of ETS. As a consequence, not a single organization, not even the Better Business Bureau (BBB) can hear or audit the recording when a test taker requests to prove transparency of the exam.

Accordingly, this seems highly unfair for test takers when they are suspicious about their recording. When they ask to disclose the recording, ETS responds that it cannot give parts of speech or run the recording to prove transparency of the Speaking section because of security reasons. This is ridiculous and it shows that ETS uses monopolistic power to take advantage of test takers. The causes of failure might be the low volume of their voice, or technical errors resulting from defective equipment, or incomplete voice transmission resulting from static, or the echo of a microphone or Internet transmission errors. As you may know, voice transmission through the Internet is not comparable to true voice as in a face-to-face interview.

In IELTS, after an interview, the interviewer will replay the tape to make sure that the recording is done properly. If not, another interview or exam will be done immediately until the tape is satisfactorily recorded. Moreover, IELTS clearly states that the grading is done during the interview, not from the recording. The recording is used only as a reference in case that a candidate requests to check for transparency of the exam.
We are told by test proctors that we should speak slowly and loudly in the test so that the recording will be better recorded. In real life, we do not have to worry about speaking to the microphones when we deliver a presentation or have a conversation with our peers or professors or patients and we do not have to be sure that our voice will be recorded properly. Most importantly, we can repeat or correct our speech immediately without worrying of time limit if we make mistakes.

4.2 Vague Evaluation

4.2.1 Useless Evaluation

We are very disappointed to receive the canned evaluation in which ETS copies and pastes the detail of range of score evaluation of 1.5-2.5, 2.5-3.0, and 3.0-4.0 for every two questions rather than giving us the individually detailed evaluation for each question. The evaluation sheet given to test takers is too general and too vague. For example, when the evaluation says that the speech is difficult to understand because of pronunciation or grammatical mistakes, the raters should give specific examples of part of test takers’ speech. For example, some speech patterns may be interpreted as an accent rather than mispronunciation. It depends on the ability of the raters to interpret the speech whether as to it is acceptable or not.

4.2.2 Identify a phrase with the problem

Identifying sentences or phrases with the problem, we strongly believe, is crucial and necessary. Pointing out which part of delivery is unacceptable or wrong is a key for the test takers’ success. Otherwise, they will continue to make the same mistakes without knowing that their mistakes are so severe that the raters will punish them. This causes serious frustration to test takers who spend time, money, and effort hoping to receive a desired score to improve. We do not know exactly where to improve. We all have to read raters’ minds. This is unfair for test takers and causes emotional distress to test takers and their families. If ETS is willing to improve testing practice, giving a clear evaluation is a must. Therefore, if ETS cannot give a specific evaluation to test takers, it should not include the Speaking section in the TOEFL iBT at all.

To make it clearer, please see the enclosed document that demonstrates the differences between the evaluation given in the real test, which is given the same for everyone who receives the same score range, and the copied evaluation published in its official guide sold in the market, which explains differently for each individual.

4.3 Is TOEFL iBT speaking a valid test for the real world?

According to ETS (ETS: Converting Rubric Scores to Scaled Scores, 2004, ETS: Score Data Summery, 2007), to receive a score of 26, test takers must receive at least 3.33 out of a rubric score of 4 which is very high. In addition, it is speculated that to receive a score of 26, a candidate must be able to be at about 85 percentile. It means that when ranking all candidates, the successful candidate who receives higher than a score of 26 must be at about the top 15 percentile.

As a result, this practice raises questions about a possible conflict of interest for ETS and the TOEFL iBT test design. We ask ourselves whether the questions are too difficult or there is not enough time given to organize our response, or are the questions even appropriate given the vast cultural diversity of candidates.

Keeping this in mind, some TOEFL iBT test takers have received advanced degrees from American universities, have worked for universities and companies successfully and continue to receive a score lower than 26 on Speaking section of TOEFL IBT. In addition, others have responded that even though English is their primary language, they have lived in the U.S. for several years, and communicate exclusively in English on a daily basis; they continue to score below 26 on the Speaking section of TOEFL iBT. This indicates that the TOEFL iBT Speaking may not be a valid test for the real world.

4.4 Scoring Inconsistencies and Errors

Furthermore, inconsistencies with scoring occur very often. ETS allows test takers to rescore the Speaking section of TOEFL iBT by paying $60. It is discouraging when candidates receive a score lower by several points or even up to 5 points. We feel that the discrepancies of score errors are too high and the grading system is not reliable and depends on the mercy of the raters. It seems that rescoring system becomes another source of income for ETS.

5. Registration and Customer Feedback

5.1 Availability of TOEFL iBT and Improper Testing Schedule

Another frustration test takers have is that ETS is not able to provide a test at a good test center at a convenient time and location. At many locations in the United States, all test centers are occupied a few months before the test dates. Test takers must register at least a few months ahead at a location they sometimes are not satisfied with as there are no available seats in the desired location.
Indeed, this information can be confirmed by Wikipedia, the free encyclopedia (Wikipedia, TOEFL iBT, 2007):
Candidates complain about the TOEFL iBT, mainly because of the high noise level during the speaking section: Everyone is responding orally to six questions at the same time. In addition, the number of seats available is limited (usually filled four months before deadline date)[citation needed] (Wikipedia, 2007).
Inside Higher Ed, the online source for news, opinion, and jobs for all of higher education (Jaschik) announced that access is a problem for TOEFL candidates:
Since ETS introduced the internet based TOEFL in September 2005, thousands of students all over the world have endured immense stress, inconvenience and financial loss: waiting months to get a seat, traveling far to testing centers, and then having tests postponed or cancelled at the last minute. Why? Because there are not enough test centers. Why not? Because ETS does not offer institutions enough of a financial incentive to become a test center. Moreover, the infrastructural support offered to the few existing test centers is weak.
On the other hand, I wish that ETS had paid attention to the suffering of international students and fixed the administrative problems of the TOEFL iBT. If they had done that, they would not be running into these access problems now. ETS, please invest some of your millions into infrastructure and access (Priscilla Allen, 2007, April 4).
In addition, ETS continues to offer the test at inappropriate time such as 4 a.m., 6 a.m., 6 p.m., 7 p.m. and 10 p.m., which are the times that many people are supposed to rest.
When there are no test centers left, test takers have no choice. They must register for a test and take it. These inappropriate time schedules affect test takers’ performance, which result in poor scores on their tests.

5.2 Customer Response Failure

After facing on-going problems, many members feel that ETS continually ignores their inquiries regarding score evaluations and the poor testing environments. When emails or letters are sent to ETS advising them of these problems, a “canned” response is what we get. The problems are not addressed. Often, we never receive response.

6. Number of test questions and total time differences among test takers.

It is unfair that test takers may be required to perform different amounts of tasks and a different number of questions on the reading and listening section. Candidates may receive either three or five passages during the reading section and six to thirteen passages of listening section. We think that everyone should get the same amount of questions. We were informed that ETS gave dummy questions which were not scored for some test takers. These dummy questions are probably for ETS's exam trial. This exhausts the test takers. This practice should not be allowed.

Test takers are exhausted after reading five passages for the reading section before doing the Speaking section. Test takers are not allowed to go to the rest room during any sessions. It is impossible to go to the rest room during the reading because the reading is divided into two to three subsections:

1 reading passage ---20 minutes
2 reading passages-- 40 minutes
2 reading passages ---40 minutes

Furthermore, average waiting time for registration process is about 1 hour. Occasionally, test centers encounter technical or server problems which result in longer waiting time. Test takers might have to wait two to three hours to complete registration process. After that, they have to complete the reading and listening section before they can take a break. We think that test takers should be allowed to take a break after they finish 60 minutes of the reading section or before they go on the listening section. We feel that these issues should not be taken lightly because they involve health-related problems, such as urinary tract infection.

04-10-2008, 05:20 AM	#7 (permalink)
knok TestMagic Guru Moderator Join Date: Nov 2003 Location: USA Posts: 3,045	TOEFL iBT Complaint: Subject: TOEFL iBT Complaint June 19, 2008 Dear_____ We are a group of the Test of English as Foreign Language (TOEFL iBT) test takers who are concerned with the test practices of Educational Testing Service (ETS). We would like to express our frustration about the testing center availability, testing environment and equipment, and test scoring process. ETS has published many reports and official guide to give an impression that the TOEFL iBT is given fairly. However, the speaking test is too subjective and the evaluation is too vague for test takers. Hundreds of test takers continue to have bad experiences with TOEFL iBT test and services while they have paid a high testing fee of $150 for each test. We would like your office to help us with this situation. TOEFL iBT Complaint 1. Defective and Low-Standard Equipment Overall, Thomson Prometric testing centers are bad, others are worse. 1.1 Computer Monitor Some Thomson Prometric testing centers are using old monitors, which are not the LCD type. Some computers used in this test center should be thrown away since they do not provide good images anymore. They are hard to read because they blink. Some computers are not set to have high contrast between images and background. This makes it hard for test takers to read the passages or, if they can read them, they will have to stare at the screen intensely and hurt their eyes. Particularly, test takers are required to read perhaps five passages on the reading section, for a total time of two hours for the reading section alone. This problem usually occurs at Thomson Prometric testing centers. When test takers asked to change their monitors, the test proctors responded that this was the way they were set up and they had nothing to do with them and suggested test takers to write the complaint to ETS directly. Unfortunately, nothing had been resolved after several hundred test takers filed their complaints to ETS. This problem has continued for years. We suggest that LCD monitors, which offer the latest technology be used for the current TOEFL iBT. This type of monitor can be acquired at less than $170 each. 1.2 Internet Connection Errors or Computer Problems In addition, in some test centers, it takes too long to see another question after clicking the answer on the previous question. It is not fair for test takers because this test is giving in a time constraint. On some occasions, test takers cannot see some buttons on the test or they cannot change the answers after clicking on the answers. It may be true that test takers can raise their hands and request a test proctor to stop the test and re-operate it but in some occasions it is impossible because test proctors do not see test takers when they need help immediately. Test takers waste their test time at least 5-10 minutes asking for help. Also, in the reading section, the information that test takers already read before calling test proctors is lost from their memory. They must re-read it again. This results in a lower score on that section. This problem usually happens at some other testing centers that ETS uses, such as high schools or colleges. These facilities have limited hardware or Internet capacity. Most importantly, they set tables too close to each other with no partition, less than one meter apart. 1.3 Insufficient Paper Notes and Pencils Furthermore, test takers must take notes during the test. Some test centers give only a few pieces of paper to test takers and tell them to ask for paper later when candidates use it up. When candidates run out of paper during the test, it is almost impossible for them to ask for paper in the middle of the test. This is because it is not convenient and time consuming to ask for paper. Also, some test centers do not provide enough sharp pencils to candidates. Writing with a dull pencil will reduce the speed of taking notes which reduces ability of test takers to perform well, especially on the Speaking and Writing Section. 2. Disturbing Testing Environment 2.1 Noisy Environment Most importantly, the test environment should be quiet at all time. Nobody should be allowed to speak in a test room at any time. First, when people are testing microphones at the beginning, some test takers are working on the reading section. It is difficult to focus and perform well in answering questions. Second, when some test takers are listening to the passages on the listening section, others are shouting to answer the six speaking questions. Remember that test takers are checked in at different times so the time of speaking may take longer than 40 minutes. It is hard to concentrate on listening or writing when they must be in distracting environment. 2.2 Unprofessional Test Administration Moreover, the test proctors, who should be ensuring that we have a proper environment for taking the test, are sometimes the biggest problem. They have been known to carry on conversations while the test is in progress. Some test proctors even sign candidates in, take pictures of candidates inside the test room, surf the Internet next to test takers while the test is in session, and even listen to music without headphones. 3. Format and Time Constraint in the Speaking Section ETS states: The raters who listen to your response will analyze it in three general categories. These categories are Delivery, Language Use, and Topic Development. All three categories have equal importance. Use the sample Independent and Integrated Speaking Rubrics on pages 242 to 246 to see how responses are scored. This section includes important points that should be covered when answering each question. All of these points must be presented in a response in order for it to receive the highest score in the Topic Development category. These important points are guides to the kind of information raters expect to hear in a high-level response. (ETS official guide to the new TOEFL iBT, p.233) ETS also published the strategies for raising the TOEFL Speaking score by telling test takers to ask themselves these questions: Did I complete the tasks? Did I speak clearly? Did I organize my ideas clearly and appropriately? Did I provide a complete response? Did I use the time effectively?” (ETS official guide, p. 227) The ETS official guide recommends candidates to time themselves when practicing the speaking tasks so that they can get an idea of how much can be said in the allotted time (ETS official guide, p. 247). Officially, in the Speaking Section, there are only six questions. The first two questions are called Independent Speaking Tasks because they require test takers to draw entirely on test takers’ ideas, opinions, and experiences when responding. Test takers have 15 seconds to think and 45 seconds to speak. It is really difficult for test takers to think and formulate their thoughts in 15 seconds and organize and speak in 45 seconds, particularly when the questions are not familiar in their daily lives. For the rest of the questions, candidates are required to summarize the conversations, lectures, and reading provided. In some tests, the topics are too difficult or are not in the interest of test takers. This test is unrealistic for measuring speaking ability of test takers. It is the test that puts more focus on organizing speech in the time limit. ETS mentions on its ETS official guide that test takers may not finish the tasks but can still get a high score and, in the meantime, those who can speak fluently may not get a high score in the Speaking section test if they cannot answer intelligibly and sustainably. Not surprisingly, after asking many test takers, even though they can speak fluently, most of them are very nervous during the test and cannot finish answering the questions, none of them can get a score higher than 24. ETS informs institutions that a score of 26 is a score that is equal to a score of 50 on the Test of Spoken English (ETS official guide, p. 350). However, a score of 26 on the speaking section of TOEFL iBT is currently a minimum score for professionals to fulfill their requirements for their professional licenses. According to TOEFL® Test and Score Data Summary for TOEFL Internet-Based Test: September 2005-December 2006 Test Data (ETS, page 9-10), the mean score of the candidates from the United States on the Speaking section of TOEFL iBT was only 22 whereas the mean score of the candidates from countries in which English is their primary language is 23. Even though some of these Americans may not have grown up speaking English in America, it is important to recognize the relatively low score for those who have been in an English speaking country for their entire lives. This result indicates that the test is invalid and unrealistic in the real world. This test seems to measure intelligibility or other aspects rather than the language ability. This seems to be unfair for test takers when it is obvious that getting a high score on the Speaking section has nothing to do with the ability to speak English fluently. 4. Grading System: 4.1 Transparency of Recording In TOEFL iBT, in the speaking part, a narrator tells candidates to check the microphone and one warm-up question is given to check its system, but how can we know that the volume and clarity of our voice is sufficient to be understood or clear enough to produce a good impression on the raters? Test takers will never know whether their recording is done properly or not because we cannot hear our own response. Obviously, ETS has candidates sign a document before taking the test stating that after finishing the test, all exam results or recordings will solely become a property of ETS. As a consequence, not a single organization, not even the Better Business Bureau (BBB) can hear or audit the recording when a test taker requests to prove transparency of the exam. Accordingly, this seems highly unfair for test takers when they are suspicious about their recording. When they ask to disclose the recording, ETS responds that it cannot give parts of speech or run the recording to prove transparency of the Speaking section because of security reasons. This is ridiculous and it shows that ETS uses monopolistic power to take advantage of test takers. The causes of failure might be the low volume of their voice, or technical errors resulting from defective equipment, or incomplete voice transmission resulting from static, or the echo of a microphone or Internet transmission errors. As you may know, voice transmission through the Internet is not comparable to true voice as in a face-to-face interview. In IELTS, after an interview, the interviewer will replay the tape to make sure that the recording is done properly. If not, another interview or exam will be done immediately until the tape is satisfactorily recorded. Moreover, IELTS clearly states that the grading is done during the interview, not from the recording. The recording is used only as a reference in case that a candidate requests to check for transparency of the exam. We are told by test proctors that we should speak slowly and loudly in the test so that the recording will be better recorded. In real life, we do not have to worry about speaking to the microphones when we deliver a presentation or have a conversation with our peers or professors or patients and we do not have to be sure that our voice will be recorded properly. Most importantly, we can repeat or correct our speech immediately without worrying of time limit if we make mistakes. 4.2 Vague Evaluation 4.2.1 Useless Evaluation We are very disappointed to receive the canned evaluation in which ETS copies and pastes the detail of range of score evaluation of 1.5-2.5, 2.5-3.0, and 3.0-4.0 for every two questions rather than giving us the individually detailed evaluation for each question. The evaluation sheet given to test takers is too general and too vague. For example, when the evaluation says that the speech is difficult to understand because of pronunciation or grammatical mistakes, the raters should give specific examples of part of test takers’ speech. For example, some speech patterns may be interpreted as an accent rather than mispronunciation. It depends on the ability of the raters to interpret the speech whether as to it is acceptable or not. 4.2.2 Identify a phrase with the problem Identifying sentences or phrases with the problem, we strongly believe, is crucial and necessary. Pointing out which part of delivery is unacceptable or wrong is a key for the test takers’ success. Otherwise, they will continue to make the same mistakes without knowing that their mistakes are so severe that the raters will punish them. This causes serious frustration to test takers who spend time, money, and effort hoping to receive a desired score to improve. We do not know exactly where to improve. We all have to read raters’ minds. This is unfair for test takers and causes emotional distress to test takers and their families. If ETS is willing to improve testing practice, giving a clear evaluation is a must. Therefore, if ETS cannot give a specific evaluation to test takers, it should not include the Speaking section in the TOEFL iBT at all. To make it clearer, please see the enclosed document that demonstrates the differences between the evaluation given in the real test, which is given the same for everyone who receives the same score range, and the copied evaluation published in its official guide sold in the market, which explains differently for each individual. 4.3 Is TOEFL iBT speaking a valid test for the real world? According to ETS (ETS: Converting Rubric Scores to Scaled Scores, 2004, ETS: Score Data Summery, 2007), to receive a score of 26, test takers must receive at least 3.33 out of a rubric score of 4 which is very high. In addition, it is speculated that to receive a score of 26, a candidate must be able to be at about 85 percentile. It means that when ranking all candidates, the successful candidate who receives higher than a score of 26 must be at about the top 15 percentile. As a result, this practice raises questions about a possible conflict of interest for ETS and the TOEFL iBT test design. We ask ourselves whether the questions are too difficult or there is not enough time given to organize our response, or are the questions even appropriate given the vast cultural diversity of candidates. Keeping this in mind, some TOEFL iBT test takers have received advanced degrees from American universities, have worked for universities and companies successfully and continue to receive a score lower than 26 on Speaking section of TOEFL IBT. In addition, others have responded that even though English is their primary language, they have lived in the U.S. for several years, and communicate exclusively in English on a daily basis; they continue to score below 26 on the Speaking section of TOEFL iBT. This indicates that the TOEFL iBT Speaking may not be a valid test for the real world. 4.4 Scoring Inconsistencies and Errors Furthermore, inconsistencies with scoring occur very often. ETS allows test takers to rescore the Speaking section of TOEFL iBT by paying $60. It is discouraging when candidates receive a score lower by several points or even up to 5 points. We feel that the discrepancies of score errors are too high and the grading system is not reliable and depends on the mercy of the raters. It seems that rescoring system becomes another source of income for ETS. 5. Registration and Customer Feedback 5.1 Availability of TOEFL iBT and Improper Testing Schedule Another frustration test takers have is that ETS is not able to provide a test at a good test center at a convenient time and location. At many locations in the United States, all test centers are occupied a few months before the test dates. Test takers must register at least a few months ahead at a location they sometimes are not satisfied with as there are no available seats in the desired location. Indeed, this information can be confirmed by Wikipedia, the free encyclopedia (Wikipedia, TOEFL iBT, 2007): Candidates complain about the TOEFL iBT, mainly because of the high noise level during the speaking section: Everyone is responding orally to six questions at the same time. In addition, the number of seats available is limited (usually filled four months before deadline date)[citation needed] (Wikipedia, 2007). Inside Higher Ed, the online source for news, opinion, and jobs for all of higher education (Jaschik) announced that access is a problem for TOEFL candidates: Since ETS introduced the internet based TOEFL in September 2005, thousands of students all over the world have endured immense stress, inconvenience and financial loss: waiting months to get a seat, traveling far to testing centers, and then having tests postponed or cancelled at the last minute. Why? Because there are not enough test centers. Why not? Because ETS does not offer institutions enough of a financial incentive to become a test center. Moreover, the infrastructural support offered to the few existing test centers is weak. On the other hand, I wish that ETS had paid attention to the suffering of international students and fixed the administrative problems of the TOEFL iBT. If they had done that, they would not be running into these access problems now. ETS, please invest some of your millions into infrastructure and access (Priscilla Allen, 2007, April 4). In addition, ETS continues to offer the test at inappropriate time such as 4 a.m., 6 a.m., 6 p.m., 7 p.m. and 10 p.m., which are the times that many people are supposed to rest. When there are no test centers left, test takers have no choice. They must register for a test and take it. These inappropriate time schedules affect test takers’ performance, which result in poor scores on their tests. 5.2 Customer Response Failure After facing on-going problems, many members feel that ETS continually ignores their inquiries regarding score evaluations and the poor testing environments. When emails or letters are sent to ETS advising them of these problems, a “canned” response is what we get. The problems are not addressed. Often, we never receive response. 6. Number of test questions and total time differences among test takers. It is unfair that test takers may be required to perform different amounts of tasks and a different number of questions on the reading and listening section. Candidates may receive either three or five passages during the reading section and six to thirteen passages of listening section. We think that everyone should get the same amount of questions. We were informed that ETS gave dummy questions which were not scored for some test takers. These dummy questions are probably for ETS's exam trial. This exhausts the test takers. This practice should not be allowed. Test takers are exhausted after reading five passages for the reading section before doing the Speaking section. Test takers are not allowed to go to the rest room during any sessions. It is impossible to go to the rest room during the reading because the reading is divided into two to three subsections: 1 reading passage ---20 minutes 2 reading passages-- 40 minutes 2 reading passages ---40 minutes Furthermore, average waiting time for registration process is about 1 hour. Occasionally, test centers encounter technical or server problems which result in longer waiting time. Test takers might have to wait two to three hours to complete registration process. After that, they have to complete the reading and listening section before they can take a break. We think that test takers should be allowed to take a break after they finish 60 minutes of the reading section or before they go on the listening section. We feel that these issues should not be taken lightly because they involve health-related problems, such as urinary tract infection. _ _ _ _ SIG _ _ _ _ "Every reason that tells you that things will never change disappears." "I did what I had to do because it is the right thing to do." Last edited by knok : 04-10-2008 at 05:42 AM.