University at Buffalo - The State University of New York
Skip to Content

English Communication Skills Assessment

COMPARISON OF TWO PROCEDURES USED TO SCREEN PHARMD APPLICANTS FOR ACCEPTABLE ORAL ENGLISH COMMUNICATIONS SKILLS

Robert G. Wahler, Jr., PharmD, CPE; Erin Slazak, PharmD, BCPS; Christine Stumm, MS; Jennifer Rosenberg, PhD

Background

Doctor of Pharmacy (PharmD) students with poor oral English communication skills often perform poorly during Experiential Education activities and simulated patient counseling scenarios, and may ultimately expose patients to inadequate medication counseling leading to medication misuse.  During the 2014-2015 admissions cycle, the University at Buffalo School of Pharmacy and Pharmaceutical Sciences (UB SPPS) implemented PharmSpeak, a mock patient counseling session held in conjunction with candidate interviews. Faculty assessment of the PharmSpeak sessions was communicated to the Admissions department. During the 2015-2016 admissions cycle, UB SPPS implemented an Oral Communication Rubric (OCR) to screen candidates during both their interview and PharmSpeak session.  A score of ≤6 out of 9 triggered the school to require the candidate take the Oral Proficiency by ComputerÒ (OPIc) test.  A minimum result of “Advanced Low” was required for admission.  Due to the labor-intensive nature of PharmSpeak, the goal of the school was to shift the screening to the faculty/staff/alumni interviewers if both processes were equally effective at flagging students with questionable oral English communication skills.

Objectives

To determine whether an interview-based screening procedure yielded similar results in identifying students with questionable oral English communication skills as did the PharmSpeak screening procedure.

Methods

During the interview cycle of 2015-16, the OCR was integrated into the process.  Candidate interviewers were combinations of UB SPPS faculty, staff, and alumni with or without a current student interviewer.  Interview team members were oriented to the OCR prior to each interview session and completed it for each candidate.

Faculty members (RGW, ES) administered the PharmSpeak program using video capture in the UB SPPS Patient Assessment suite and completed the OCR.

Candidates scoring ≤6 on the 9 point rubric either by interviewers or by PharmSpeak faculty were referred to the admission department for completion of the OPIc.

Demographic data on applicants were collated with OCR scores and OPIc scores.

Data were analyzed using a chi-squared analysis to determine if both methods of assessment were able to flag candidates at similar rates.

This study was determined to be “Not Humans Subjects Research” by the University at Buffalo Institutional Review Board, IRB ID: STUDY00000871

Discussion

Our primary objective was accomplished as we were able to show that both uses of the OCR performed without a detectable difference between applications. Data were analyzed using a chi-squared analysis to determine if both methods of assessment were able to flag candidates at similar rates. The study had 70% power to detect an increase from a predicted 10% “flag” rate (PharmSpeak) to a 15% rate for the interview-based assessment.  Based upon these results, we have eliminated the PharmSpeak exercise as part of the interview process.

Of those flagged by only 1 method (N=16), none were identified by the OPIc to be below admission criteria (OPIc < Advanced Low per the Technical Standards of the UB SPPS).  More importantly, both methods successfully identified those candidates (N=5) that were below admission criteria.

A limitation of the study was the relatively low power.  As we have eliminated the PharmSpeak portion of the interview process, increasing the number of candidates evaluated by both methods to improve the power of our dataset is impossible.

Additionally, the number of candidates flagged by only 1 method raises concerns of subjectivity by the users.  The OCR was created as an internal tool to standardize our screening process, yet it has not been validated.  Further study would be necessary to improve the value of the OCR.

Three flagged candidates did not take the OPIc.  One was identified by interview as a native English speaker who was so quiet that interviewers found scoring the OCR challenging. One candidate was interviewed via Skype by both admission staff (JR) and faculty (ES). The PharmSpeak exercise was impossible to conduct; but the candidate did not pass the full interview thus obviating the need for further communication evaluation. The third candidate was interviewed prior to full implementation of the OPIc process.

Previous reports on pharmacy school admissions processes are limited regarding oral English communication proficiency.1,2  Because of the diversity of our candidate pool, it is imperative that we have processes in place to balance the desires of the candidates to become pharmacists with the need for practicing pharmacists to effectively communicate with patients and health-care personnel.  The combination of the OCR with the OPIc allows us to do so.

Conclusion

Use of the OCR during the interview did not differ significantly from the use of the OCR applied to the PharmSpeak session.

"); pageTracker._trackPageview();