Multilingual Testing in Monolingual Regimes

Shyam Sharma

Shyam Sharma

A few months ago, while attending a talk on campus (at Stony Brook University) one afternoon, given by Elana Sohamy, an Israeli scholar, I had a moment of despair.

The title of her talk was “multilingual testing” and the backdrop of her presentation was the monolingual regime of language testing and its effects on multilingual language users across the world.

As teachers of language and writing/communication, we keep saying in theory that language learners take 3-5 or even 9-11 years to be fluent and accurate in a new language, depending on where and how they learn. But in practice, we continue to resort, very quickly and thoughtlessly, to the logic of pragmatism, of institutional policy, of the need to make sure that our multilingual students can perform in English.

Here’s the problem with our dead habits and thoughtless embracing of the current monolingual testing regime: Usually, monolingual tests don’t predict overall academic performance by multilingual students, simply because the tests confuse language proficiency as a predictor of a lot more than just language proficiency that academic transition and success involve. It would be nice if, for instance, a TOEFL score of x predicted whether a graduate student is able to listen, read, speak, and write well enough to be a successful graduate student in, say, a graduate program in ecology. But it doesn’t.

Two students with the same TOEFL score (even with similar academic records from the same education system) regularly perform significantly differently, and often, students with much lower TOEFL scores perform better than with much higher scores. Why? Because academic performance is a matter of process involving a lot of factors–grit, support, psychology, personality, content knowledge, and a lot more.

While TOEFL is not as bad as GRE, I must add, the company selling this product back-peddled a few years ago when it ignored that adding the spoken test (as it is administered by using an underdeveloped technology of speech and accent recognition) makes the test even less valid. TOEFL’s test of writing skills is as much of a joke as SAT writing test is: it rewards certain semantic, syntactic, and rhetorical tricks that do not constitute good writing in college/university. And while its listening and reading passages are based on quite authentic classroom situations, the validity of the overall score is significantly hampered by the outdated view of language proficiency that the test is based on. Those who make these tests are still unable to understand how multilingual English speakers around the world perform sociolinguistically in academic and other contexts.

I heard today that ETS is starting to respond to the basic idea that multilingual English speakers should be assessed in terms of how they draw on more than one language to achieve communicative goals. But the company’s primary goal of efficiency for itself trumps the principles of validity and effectiveness. And this is to say nothing about the blatant lie about universality of the content on which the tests are based, the egregious amount of money the test fee translates from US dollars to some of the local currencies, the intellectual insult for students with learning disabilities, and many more issues.

Sohamy showed that immigrant students scoring in the 60th percentile when tested in L2 only were able to score in the upper 80s when the questions were provided in both L1 and L2. This means that when educators allow learners to start academically succeeding while still having some “issues” with their language, they learn English much more effectively in the process. This is a no brainer: when I start a semester, I tell my nonnative English speaking students in class that they “shouldn’t worry about [their] language” and instead “focus on doing the research, coming prepared to actively participate in class, drawing on [their] prior knowledge, being excited about learning and sharing ideas.” To teachers, it is really a no brainer: It is possible not to put the cart of language learning in front of the horse of the process of education. It is possible not to follow the backward logic of ETS — of treating language as something that you learn “before” you join the learning party!

Gatekeeping is necessary but it doesn’t need to be so outdated and invalid. Assessment is necessary but it doesn’t need to be decoupled from learning and teaching.

Now, some readers may object: “I have to make sure that students I admit have a certain level of language proficiency.” Well, there are two significant problems in that “pragmatic” stance. First, someone who scored well on the TOEFL may have been linguistically privileged and proficient, but there is no guarantee that he/she is academically capable or committed. TOEFL doesn’t measure subject knowledge, and, again, it doesn’t measure grit. Second, one could say that admission officers look at academic transcripts in order to review the applicant’s academic caliber. Guess what? Academic transcripts from different countries (and even from different academic systems from the same country) cannot be compared–which is one of the reasons people turn to TOEFL in the first place. Back to square fifteen.

We’re confusing pedagogy with policy, process with desirable proficiency, outcome with entry level proficiency, and our own bias with the need to rely on a system that we admit is flawed but seemingly without good alternatives. What if start by thinking outside of these easy frameworks? What if we start by embracing what Elana Sohamy called “critical language testing”–testing based on skepticism toward established regime that are not in the business of fairness and sophisticated thinking. What if we can adopt parallel regimes, including ad hoc approaches, that we can use in order to challenge ourselves?

What if we can ask all the ten students whom we want to admit to our doctoral program to call us on Skype, Hangout, Viber, or Facebook phone and have a twenty minute conversation each, in order to use our own conscientious judgment, instead of a TOEFL score?

What if?

 Shyam Sharma is, PhD in rhetoric and composition from University of Louisville, is assistant professor of writing and rhetoric at State University of New York, Stony Brook.  

Giving a Benefit of Doubt During Assessment

Umes Shrestha

I am a teacher who likes being very liberal while checking answer papers and in this article I am going to talk about why I detest our testing system; why I give a benefit of doubt to the students; and why doing so, I believe, helps foster their confidence and reinforce their learning. I’d like to start by talking about how difficult it is for me to fare with tests and exams, and suggest that we stop doing to our students what many of us as teachers can’t bear ourselves.

Besides teaching, I am also pursuing a Master’s degree in Education. It is a semester system, which means there’s a final examination every six months. And I find it really stressful; in fact I have always found examinations very stressful throughout my life. I may not know my strengths but I know one of my weaknesses for sure – I cannot read books or notes to memorize answers. I do understand the overall concepts but I can’t memorize them in order to spill them all over the answer sheet. I can’t do it even if my life depended on rote memorization. Far worse, my attention span is shorter than an ant’s tail. I cannot remain calm for more than 10 minutes. I am always drowning in a sea of distractions. Family. Friends. Facebook. Music. Home. Job. Deadlines. I can’t quite keep up the concentration. I get itchy and I have to take a short break every 10 minutes or so. My mind operates that way.

While taking exam, I can’t think fast enough to frame and organize my answers. I need time to generate ideas, plan a structure and write essays. I make a lot of spelling errors because I heavily rely on spell checkers. My handwriting and presentation look impeccable in the first two pages but they soon start deteriorating as the clock in the exam hall ticks away in a flurry.

That’s my story. I don’t like being tested in such constrained time and in such ominous hall. I feel it is completely unfair to be judged on what I write in three hours. And to add to my frustration, I am always haunted by the usual mentality of language teachers who are very strict while checking answer papers. And I’m pretty sure there are many out there, both teachers and students, who can perfectly relate to me. Therefore I believe this extreme emphasis that our educational system puts on summative testing as the only mode of assessment could be very damaging to our students and, as a consequence, to our society at large.

So let me ask you to do this: imagine yourself being a teenager, surrounded by technology and media, having access to unlimited knowledge and resources, having friends who are hyperactive on the internet and social networking sites, having too many options at hand. Imagine yourself sitting in the class for hours and hours, sitting on the same desk and ‘listening’ to the teachers who use the same deadbeat methods day in and day out. Imagine yourself not knowing (or doubting) the relevance of the traditional education, getting certificates, getting degrees, joining the workforce and doing the same old jobs.
exam-makes-no-sense

(Pic: a screenshot of one of my student’s facebook status. This student is perfectly clueless.)

And we ask them to sit for exams. We ask them to write an essay on festivals of Nepal. Or we tell them to compose a speech on Nepal’s historical figures. We tell them to write a complaint letter with date, salutations, body and closings in a precise format. We tell them to write coherently and cohesively. And we want them to write without any spelling errors.

Worst of all, we don’t try to find alternatives to the discouraging test-driven system. Instead, we make the overall process and environment of learning frightening for students by using the fear of exam to work like slaves throughout the year, by focusing on what they lack all the time, and by being strict when assessing their work.  And at the end, even when we could have done otherwise, could have at least made a better combination, we just tell them to sit for three hour exams and expect them to compose perfect essays with their perfect handwritings. We stress on neatness and cleanliness; we write comments, underline wrong answers, we circle misspellings and with red ink we cross out answers that don’t match our expectation. That’s what we usually do, don’t we? There could be different ways to assess our students but at the end of the day, we (have to) judge students based on written examinations.

It would seem as if in the present context, there is absolutely no way out of this final written examination module but dear teachers, here are some of the things I do, and as a teacher, this is what I propose you to try out these and similar things as well.

I give my students a benefit of doubt while checking their answer papers. I understand, they don’t take exams because they like it. I have never come across any student who loves taking exams. On top of it, they have to take exams of seven or more subjects every term. I can perfectly relate to the maddening pressure to perform. They have no option, and neither do we. So while I check their answer papers, I gently remind them of their spelling mistakes but don’t deduct any marks for it. I overlook minor errors and slips like she don’t have friends. If they write any brilliant essays with a logical framework, I get super joyous about it. And I give it 10 out of 10. Why not! (This one student glared me back with disbelief and said: Sir, you gave me 10 out of 10? And I said: Yes, you deserved it.) If their essays look messy and are riddled with terrible structures, I don’t butcher their effort thoughtlessly. They would have done it better if they didn’t have any time constraints or if they had time to draft and re-draft the essays. I try to be sensible about their handwriting too, because writing for three hours straight is a real pain in the ass and the wrist. I go through the same pain every time I take an exam. One reason, many of us are used to typing on computer rather than writing with a pen. And I am never picky about grammatical errors. I have seen my professors at the university make grammar errors. I have seen me making horrible grammatical errors. We all do.

I try to keep track of the students, their progress, their assignments, classroom participation, portfolios – but I don’t judge them based on one written examination. I am definitely not seeking an easy way out. I communicate with the students who are lagging behind and make them feel safe. I do demand high, push their limits but I want to be realistic, and set realistic goals and realistic expectation. At the end of the day, I feel like I have done my part and go to bed without any resentment or bitterness.

Our testing system is full of holes, and it desperately needs a facelift. However, we can’t change the entire system right away, let’s be realistic. But what we can do is empathize with the students. Therefore, dear teachers, give your students a benefit of doubt, because sometimes we too need it.

I’m going to wrap up my ‘rant’ with one more facebook status that I came across –  just to add to my argument.

umes

Umes Shrestha

M.Ed. ELT

Teaches Business Communication and Literature to undergraduate students

Language Testing and Assessment Workshop: A reflection

Ganesh Datt Bhatta

On February 23, 2014, I attended a one-day workshop on ‘Language Testing and Assessment’ conducted by NELTA in association with the American Embassy at Kathmandu University, School of Education. The workshop was facilitated by Dr. Stephen J. Stoynoff from Minnesota State University Mankato USA, one of the key note speakers for the 19th International Conference for NELTA.

stoynoff-1

It was a very lively session by Dr. Stoynoff as he presented on how learners who learn English as the second or third language struggle in learning the language in the classroom.  He shared an example of his own friend, Mr. Carlos who had immigrated to America from Nicaragua in 1950s. He belonged to a Spanish speaking community and was already 14 when he joined a school in Texas, US. It was difficult for him to learn and adjust in the classroom with his peers who were much younger by age and he did not know English either. As a result, he could not succeed in his sixth grade and he had to repeat the same grade. As English was not his first language, it was obviously difficult for him to understand what the teachers taught in English medium that followed “sink-or-swim” model.

Dr. Stoynoff asked all the trainees to find out the factors that influenced language development and reasons for Carlos’s failure in grade six. The discussion ended with findings of the factors like home environment, societal environment and individual factors of language learners influence the language development.  A learner learns language if the educational level of the family is higher. The families which give priority to learn different languages helps a child to values towards education is also important for the development in a learner. It was amazing to find that Carlos earned a PhD later, though he had tough time in school in the early years of migration. The turning point in his life was at his school where a teacher appreciated his work in Spanish. Those inspiring words from the teacher guided him to choose Spanish for his further education and he earned a PhD in the same area. Now he is regarded one of the important experts in the field of Second Language Learning and Teaching.

It can be inferred from the story of Carlos that a teacher can inspire students to pursue their career in proper field of education and the motivating environment of the family also contributes in effective learning of languages. The interaction of the factors like home environment, societal environment and individual factors of age, proficiency of the first language, attitudes towards the second language and access to the Second Language and culture affects the rate of second language development of the learners. The first session ended with one open topic to debate on the pros and cons of teaching other languages than first language at early age of learning.

The second session of the workshop focused on the current trends in the field of Language Testing and Assessment. Assessment is a process based activity to ensure that a learner is developing his/her learning that continues throughout the daily classroom activities. The trainees were divided in groups to make a student profile having some questions to check the background of students in academics performance and to design the learning environment on the bases of it to expect possible development in the pursuit of the course.

Dr. Stoynoff shared the current practices of Language Testing and Assessment in the USA and some other countries. The trainees were asked to make comparisons between the classroom tests and the standard tests (high stakes tests). The discussion gave us idea about the types of testing in classrooms and beyond classrooms in terms of their procedure of conduction.

stoynoff-3

Nepal has been following the decade-long practiced assessments and tests in all levels of education from nursery to university level for different purposes whereas the other countries have updated their testing mechanism to enhance the ability and strength of the learners. But there is no such standard test for Nepal to check the ability of Nepalese students. We came to realize that there is enough space for ELT practitioners and language experts for research in Nepal to design a standard test to check the ability of our students in Nepal vis-à-vis the students of other countries.

The one-day workshop was an eye opening program for the participants where we got to know the recent trends of language testing and assessment. I, on the behalf of all trainees would like to thank NELTA President Mr. Hemanta Raj Dahal and Ms. Sara Denne-Bolton, Regional English Language Officer (RELO) from Embassy of the USA for providing such an opportunity to discover our strengths as teachers of English language and find our students’ potentialities in learning second language comfortably.

Photos: Umes Shrestha

ganeshdatt

Ganesh Datt Bhatt
Life Member-NELTA Kanchanpur
HimalayanWhiteHouseInternationalCollege, Kathmandu
Email: yrsgdbhatt@gmail.com