[knok]

Hi AJ, IQ, Subramanian, and others,

We sent out a letter last year to NABP with no good result. AJ, about doctors, is it the same test that we mentioned in this petition?

Here is the petition to NABP.

June 19, 2007


The FPGEC
National Association of Boards of Pharmacy
1600 Feehanville Drive
Mount Prospect, IL 60056

Subject: English Proficiency Examination

Dear FPGEC:

We are a group of foreign pharmacists who would like to relay the test practice of Educational Testing Services (ETS) to the board. Some of us have taken the Internet based Test of English as a Foreign Language (TOEFL iBT) and continue to be frustrated by this test. In May 2005, we reported the problems of the Test of Spoken English (TSE) and the FPGEC suggested us to contact ETS directly. We have contacted ETS, but unfortunately, we did not receive satisfactory resolution from ETS. Presently, the TSE has been replaced by the TOEFL iBT. We feel that the TOEFL iBT is of poorer quality than the TSE, regarding testing format, testing environment and equipment, convenience of test takers, and the length of the test.

The TOEFL iBT complaint is enclosed with this correspondence. As you might note, the number of problems and deficiencies that this relatively new test has are really astonishing. Hence, we do not think that ETS will be able to improve or change the test dramatically soon. It may take years or never happen as we wait for ETS to address our needs as test takers. Not to mention the validity of the exam and conflict of interests may have caused ETS to continue with the same practice for years.

We strongly believe that it is in your hands to look for a proper and fair solution. All we are asking is the opportunity to serve our communities as registered pharmacists. That opportunity has been taken away from us since the implementations of the new TOEFL iBT requirements. You have the statistics of the number of foreign pharmacists that have already passed the FPGEE satisfactorily and have not been able to be certified only because of this language requirement. This is not an isolated case; there are thousands of us stranded in the same island.

We are sure that FPGEC is aware of many of the problems related to the TOEFL iBT. In appreciation and recognition of the importance of FPGEC’s responsibility for maintaining the high standards of practice of foreign pharmacy certification, we respectfully request that the board consider five possible solutions regarding the requirement of English proficiency testing for the FPGEE certificate program.

1. Allow other speaking test as an alternative

1.1 Oral Competency exams
We do not mind passing the speaking test, but we urge the board to use other exams if IELTS is not an option. In the state of MD, the U.S. Pharmacy graduates are required by the state to undergo an English Proficiency test (Maryland Board of Pharmacy, p. 3). They cannot be licensed in the state unless they pass the English test, even if they are native speakers. The tests on the list below offer a face-to-face interview. They are administered by “Oral Competency Vendors” (Maryland Board of Pharmacy, p. 6).

If NABP is not sure about the tests in above list, we ask NABP to consider this as a pilot program for a period of six months or so. Later on, the data collected from those who have taken these exams can be used to compare among speaking tests.

1.2 Create pharmacy English proficiency exams
In addition, for foreign physicians, the English requirement is measured through an English proficiency exam administered by the same certification authority. It is part of the three steps of the certification process of United States Medical Licensing Examination (UMSLE Bulletin, 2007):

Table 3: The Subcomponents of Step 2 CS
Integrated Clinical Encounter (ICE) Subcomponent
• Data gathering – patient information collected by history taking and physical examination
• Documentation – completion of a patient note summarizing the findings of the patient encounter, diagnostic impression, and initial patient work-up
Communication and Interpersonal Skills (CIS) Subcomponent
• Questioning skills
• Information-sharing skills
• Professional manner and rapport
Spoken English Proficiency (SEP) Subcomponent
• Clarity of spoken English communication within the context of the doctor-patient encounter (e.g., pronunciation, word choice, and minimizing the need to repeat questions or statements (UMSLE, 2007)
Basically, the English Test is a simulation of a patient-doctor interaction, as occurs in real life. We feel that physicians have to communicate exclusively and directly to patients more than pharmacists but they do not follow ETS and any other institution’s advice. What they care is that foreign physicians have to be able to communicate with their patients generally well in situations and context of healthcare, which is close to the real world. This might set an example for certifying foreign pharmacists.
2. IELTS as an Alternative

2.1 IELTS offers a face-to-face interview
Even though the IELTS is harder than the TOEFL iBT, the IELTS offers a real interview. A real interview is able to solve the problems of the recording, noisy environment, and provide two-way communication with human beings, not a machine. The common complaint about the TOEFL iBT speaking is a recording process which is too mechanical. The TOEFL iBT lacks interactive communication which is the most important part of conversation in the real world. The recording process and time constraint requiring candidates to answer a question in seconds degrade the performance of test takers who need to reach an almost perfect score of 26 on the speaking section.

In the meantime, the Wikipedia (Wikipedia, IELTS) documented that the number of IELTS users has risen from 100,000 in 1999 to half a million in 2003.
Renshaw, an English teacher (Renshaw) compared the TOEFL iBT and IELTS:
To my mind, the first of the writing tasks on IELTS clearly requires integration of reading and writing skills, and the fact that their speaking tasks involve interacting with a face-to-face interviewer (and one task where test-takers have to interact with another test-taker on-site) should lead to the general assumption that integration of skills is intrinsic to these tasks.

We shouldn't lose sight of the fact here that IELTS scores have always had a stronger correlation to actual academic ability in English-speaking tertiary contexts, and the iBT speaking section in particular was added to make the iBT more competitive with IELTS, which has had a tried and true speaking section for decades.

Don't forget also that ETS has been staring down the barrel of IELTS starting to infringe on its home territory (the United States), with dozens of US universities starting to accept IELTS scores due to its stronger reputation as a genuine measure of academic English ability in all skill areas (Renshaw, 2007).
2.2 IELTS provides a separate speaking section in an individual room
The testing conditions provided by ETS in TOEFL iBT are unacceptable. The IELTS provided a real interview in an isolated room. This can avoid the problem of a disturbing environment. Even though the recording is done during the interview, the test is graded by interviewers during a real test. The recording will be replayed to the test takers to make sure that the recording is done properly. The IELTS informed test takers that the recording is done as a reference, in case it is requested to prove the transparency. In contrast, in the TOEFL iBT, test takers have no chance to listen to their recording, no matter what.

2.3 ETS faces shortage of testing capacity
ETS has only limited available seats for the TOEFL iBT candidates, usually filled four months before deadline date (Wikipedia, 2007). As a result, alternative exams are likely to be considered in assessing candidates’ proficiency of the English language as part of the certificate program. Not to mention difficulties logging into the TOEFL.org web site when test takers want to register for tests or check their score results. It may take two weeks to log in because the web site is too busy.

2.4 Allow competition
If the IELTS or other speaking test is chosen to fulfill the board’s requirement, we believe that competition will force organizations to improve their products and services. Nowadays, the monopolistic power that the ETS holds keeps ETS from improving their efficiency and the efficacy of the testing procedure, but allows ETS to increase its profit. This situation creates unfair practice and frustration to test takers who pay the money but continue to receive this poor testing procedure. We hope that competition will change everything. Presently, it is really difficult to get a seat for the TOEFL iBT.

When a complaint has been made to ETS, it is ignored or the best resolution ETS offers is allow test takers to take another test without charge. This practice does not help a test taker because the real problems have never been solved. Test takers will not be able to improve their score if they have to take another test in the same condition and receive the same vague evaluation sheet, which requires test takers to read raters’ minds on where to improve.

3. Reduce the speaking score requirement to 20

NABP stated in the correspondence in June 2005 that, for public safety, NABP would not reduce the score requirement. Due to annoying testing conditions which degrade performance of test takers tremendously, we feel that test takers who receive a score of 20 or higher are able to communicate well in disturbing environment, time constraint, and unrealistic conversation. To receive a score of 26, test takers must receive almost a perfect score. This is unfair because the test content has nothing to do with the pharmacy field. When test takers are tested to speak on different unfamiliar topics in seconds, test takers are not able to receive a score of 26 on the speaking section because it is highly possible that they do not have an idea to answer the questions, not because they are not able to speak in English language.

For instance, if FPGEE test takers had to take an essay FPGEE exam in an annoying environment and the time was reduced to one hour, we believe that only few pharmacists would be able to receive a scale score of 75. This is not because they would be lacking pharmacy knowledge and experience but because of unfair testing conditions and not having enough time to think and organize the answer in allotted time. We do not think that it is fair for test takers when testing practice of ETS is an issue.

In addition, for those who receive a score of 20 or higher, ETS mentions in the score report that their pronunciation and grammatical mistakes do not interfere with the overall communication. They can communicate in that area. Please see the details in the copy of the score reports. This should be enough.

3.1 The mean score of American candidates was 22 on the Speaking section
According to TOEFL® Test and Score Data Summary for TOEFL Internet-Based Test: September 2005-December 2006 Test Data (Educational Testing Service, page 9-10), the mean score of the candidates from the United States on the speaking section of TOEFL iBT was only 22 whereas the mean score of the candidates from country that English is their primary language is 23. This result indicates that the test is invalid and unrealistic in the real world.

This test seems to measure intelligibility or other aspects rather than the language ability. For these reasons, we think that the current requirement is too high for foreigners whose native language is not English. Even though some of these Americans may not have grown up speaking English in America, it is important to recognize the relatively low score for those who have been in an English speaking country for their entire lives.

3.2 Is a score of 26 on the Speaking section reasonable?
In the speaking section, a score of 4 is the highest score according to ETS. To score 26 in TOEFL iBT speaking, test takers should get 20/24 for each of six questions. The highest possible score for each question that a test take is able to receive is a score of 4 so that a result will be (4*6 = 24), which is very high for any standards.

To get a score of 26, test takers should get high level score(4) in two questions and fair level scores(3) in remaining four questions, which added up to a total score of 20/24, 20/6 = 3.3 that equivalent to 26/30.

The following rating method is used by ETS:

1. (Q.1-2)  4+3 = 3.5 (GOOD), (Q.3-4) 4+3 = 3.5 (GOOD), (Q.5-6)4+3 = 3.5 (GOOD) Total Score 21/24 = 3.5 = 27/30

2. (Q.1-2)  4+3 = 3.5 (GOOD), (Q.3-4) 4+3 = 3.5 (GOOD), (Q.5-6) 3+3 (FAIR) Total Score 20/24 = 3.3 = 26/30

3. (Q.1-2)  4+3 = 3.5 (GOOD), (Q.3-4)3+3 = 3 (FAIR), (Q.5-6)3+3 (FAIR) Total Score 19/24 = 3.1 = 24/30

4. (Q.1-2)  3+3 = 3 (FAIR), (Q.3-4)3+3 = 3 (FAIR), (Q.5-6)3+3 (FAIR) Total Score 18/24 = 3.0 = 23/30

(Q.) = question number

A test taker was informed by English as a Second Language teacher (ESL teacher) who attended ETS workshops and is a member of ETS training that a score of 26 in TOEFL iBT is equivalent to 55 in the TSE, but ETS publishes in its official guide that a score of 50 on the TSE is equal to a score of 26 on the Speaking section of TOEFL iBT (ETS official guide, p. 350). This is not always true. These two tests provide different formats. They are not comparable. However, ETS suggests institutions that a score of 23 is a minimum score for International teaching assistants (ITAs) (Bowman).

The tricky part is that ETS has set a score of 26 in the speaking section as good communication skills by conducting a studying group to testify to its assumption. This studying method is criticized by educators whether it is practical in a real world or not. Please refer to Papajohn’s article. Papajohn states:

Decision makers may have to consider their own belief about the effect of passing less than qualified applicants and failing qualified applicants (Papajohn, p. 6, Faggen, p. 4).

For ITA, the mean score for panelists suggest the next generation TOEFL Academic Speaking Test (TAST) score of 23 for the cut off of the minimally acceptable speaker (Papajohn, p.5).

He also mentions that information that is conveyed in a way that is too compact may be difficult to understand. In other words, information that is too diffuse may make it harder to comprehend the connections. Response lengths of 60 seconds do not sometimes allow for correct evaluation of compactness and diffuseness (Papajohn, p. 2). He comments that no one communicates perfectly 100% of the time. The result from the ETS standard setting panel is likely to be considered as a starting point rather than as an end in itself. Rather than adopt or accept someone else’s cut score, use the standard setting process as an opportunity to raise awareness of ITA issues at their institution.

4. Count the Speaking section separately from the other three sections
We request the board to count a score of three sections as a total score by calculating from adding up all three sections as NABP required before. The total score of these three sections of 60 on the TOEFL iBT is equal to a score of 213 in the Computer Based TOEFL (ETS, 2004, p.10). It is a waste of time and money when a majority of pharmacists are able to receive a high score or passing score of the other three sections but not the speaking section to retake the total of four sections. Others have passed the TOEFL CBT but they have to retake it again.

ETS put the speaking section on the third part of the TOEFL iBT. After test takers spend three hours in the reading and listening test, they feel tired and lose the ability to concentrate because they use up their energy and spirit, especially in bad testing conditions. Some of them are so hungry that they cannot perform at their maximum potential. The ten minute break does not help reduce their stress and rejuvenate them to take the active parts, speaking and writing. Please remember that the test is operated in four to five hours or more, including waiting time for checking in.

5. Receive the highest score from the tests on different test dates or receive the total score
Since the TOEFL iBT testing conditions are unpredictable, scores may vary dramatically. For example, if the monitor blinks, test takers are not able to read from the screen clearly and fast enough in time constraint because the low quality of equipment. In addition, some test centers are too hot or too cold, making the test takers feel uncomfortable and irritated. Their reading score will be much lower than in the other tests. In the listening section and writing section, test takers are not able to hear well and concentrate on their tests while others are shouting loudly, especially when the test rooms do not have partitions or set them too close to each other.

Testing conditions affect our scores. NABP can check from test taker to test taker from their score report. If this TOEFL iBT is valid, their score is not likely to vary enormously. After asking many test takers, their scores can vary up to eight points or more. In the TOEFL iBT, in general, academic institutions require only a total of three sections, reading, listening, and writing or a total of four sections. The design of this test is likely to be counted in a total
score. In two different tests, A and B, the reading section in test A may be more difficult than that in test B but the listening section of test A may be more difficult than that of test B.

We think that a total score should be used instead of separating the score into 4 sections. In other words, a combination of the passing score should be considered to fulfill the board requirement or the use of the highest section from each test should be considered. If it is not the case, we request the board that we register and take only sections of the test we need and omit the sections we receive passing scores.

In addition to physicians, the Commission on Graduates of Foreign Nursing Schools® (CGFNS) requires foreign nurses to receive a total score of 83 for the TOEFL iBT, or a score of 725 on TOIEC, or an overall score band of 6.5 on IELTS (Academic Module) (CGFNS, 2007 Edition).

From our perspectives, nurses and doctors need to communicate with patients more than pharmacists, but their boards allow candidates to be more flexible to select the language proficiency tests. CGFNS requires candidates to pass a total score of language proficiency test rather than focus on separating score. One reason is probably that there is a high correlation among different sections of tests. In the TOEFL iBT, the speaking and writing sections become integrated tasks of four skills; reading, listening, writing, and speaking. If public safety is such an important issue that NABP prohibits candidates to use other tests to fulfill the board requirement, when compared with these healthcare professionals, we feel that this requirement should be reconsidered annually as NABP stated in 2005.

Follow Up
Finally, our group is seeking positive solutions to the concerns we have listed. It is our hope to be contributing members of society and practice our profession of pharmacy in the USA. If NABP is not able to understand our requests or if NABP does not feel that our complaints matter enough to incur changes, we are willing to meet with NABP in person. We would appreciate it if the board allows us to meet in order to explain details of our problems and difficulties. You can see that we have a thoughtful and professional request. We politely request a written response or a meeting. Thank you for your serious consideration of these issues.

Sincerely,

Your signature….

Mr./Ms.(Your name) B. Pharm (Pharmacy), M.S.,….
EE#

Enclosure: TOEFL iBT Complaint, TOEFL score reports, Test center complaint table
Cc: Competency Assessment Director, State Board of Pharmacy, Members of Congress, Governors, Attorney General

References
1. Bowman, M. S. (Nov 2005), Graduate program directors hand-out: the next generation TOEFL test-TOEFL iBT, Related issues for Speaking/TSE, ESL Program, [College of Graduate Studies, Cleveland State University]. Available at: http://www.csuohio.edu/gradcollege/f...%2011-2005.pdf Accessed June 3, 2007.

2. Commission on Graduates of Foreign Nursing School (CGFNS) Certification Program for Registered Nurses (2007 Edition), Program Requirement: English Language Proficiency. Available at: http://www.cgfns.org/files/pdf/req/cp-requirements.pdf Access June 1, 2007.

3. Educational Testing Service (2004), iBT/Next Generation TOEFL Score information, p.10. Available at: http://www.ets.org/Media/Tests/TOEFL...nformation.pdf
Accessed May 31, 2007.

4. Educational Testing Service (2007), Total and Section Score Means: TOEFL® Test and Score Data Summary for TOEFL Internet-Based Test: September 2005-December 2006 Test Data [report], p. 9-10. Available at:
http://www.ets.org/Media/Research/pd...M-0506-iBT.pdf Accessed May 29, 2007.

5. Educational Testing Service (2007) The official guide to the new TOEFL iBT. New York: McGraw-Hill, p. 350.

6. Faggen J. (1994), Setting standards for constructed response tests: an overview. Educational Testing Service. [Report]. Available at: Setting Standards for Constructed Response Tests: An Overview Accessed June 2, 2007.

7. Maryland Board of Pharmacy (2005), Test of Oral English Competency, Pharmacist Licensure application packet, p.3. Available at: http://www.dhmh.state.md.us/pharmacy...at/newexam.pdf Accessed June 2, 2007.

8. Maryland Board of Pharmacy, Oral English Competency Vendors, Pharmacist Licensure application packet, p. 6. Available at: TEST OF ORAL ENGLISH COMPETENCY Accessed May 30, 2007.

9. Papajohn, D.,(2006, June 1) Standard setting for next generation TOEFL Academic Speaking Test (TAST): Reflections on the ETS panel of international teaching assistant developers, Teaching English as a Second or Foreign Language (TESL-FJ 10.1), Volume 10, Number 1, p. 2, p. 4-5. Available at: http://tesl-ej.org/ej37/a1.pdf Accessed June 1, 2007.

10. Renshaw, J. (2007, April), TOEFL iBT and IELTS Tests Compared [Renshaw’s weblog]. April 2, 2007. Available at: iBT Speaking and Writing: News and General Gossip Accessed 2007, June 2.

11. United States Medical Licensing Examination, 2007 UMSLE Bulletin, Step 2 Clinical Skills (CS). Available at: http://www.usmle.org/bulletin/2007/examcontent.htm Accessed May 28, 2007.

12. Wikipedia (2006, June 6), The Free Encyclopedia. IELTS, Location and dates, 2007. Available at: IELTS - Wikipedia, the free encyclopedia Accessed June 6, 2007.

13. Wikipedia, The Free Encyclopedia, TOEFL. 2007. Available at: TOEFL - Wikipedia, the free encyclopedia Accessed June 1, 2007.

Note: “Dean Papajohn has worked with international teaching assistants for over twelve years teaching English as a second language classes, coordinating orientations for international teaching assistants, and conducting oral English assessment. The second edition of his book Toward Speaking Excellence has been released by the University of Michigan Press.” (TESL EJ)