Kazakhstan takes part in six international studies on assessment of school environment and educational progress of students. Moreover, the republic makes regular attempts to get students and teachers prepared for tests. According to experts, this practice cannot affect the overall score of the country significantly.
From April 4 to May 24, 2022, nearly 19 thousand students of Kazakhstan schools and colleges passed a test of functional literacy as part of PISA. This is an international comparative study of skills of 15-year-old teenagers in reading, mathematics and natural science. PISA is held every three years by the Organisation for Economic Cooperation and Development (OECD). The study started in 2000, and Kazakhstan has taken part in it since 2009.
In 2018 and 2022, the study was conducted as a computer-based test lasting for two hours. This year, assessment of reading and mathematical skills was carried out on the basis of the adaptive approach. That is, instead of fixed, pre-determined tests, participants were given tests based on their level of proficiency demonstrated at previous stages of the assessment.
“For nearly 10 years of participation in PISA, the significant proportion of 15-year-old Kazakhstani students have failed to reach the minimum threshold of literacy in three main domains of the assessment, while the indicators are constantly behind those of the OECD countries,” according to the authors of the national report based on the 2018 assessment.
The charts from the report demonstrate the gap emphasised by specialists:
According to the information-analysis centre of the Ministry of Education of Kazakhstan, PISA is “the strongest powerful research tool of the quality of school education” and “the source of reliable objective comparative data” about the preparedness of 15-year-old teenagers to successful life in modern world.
The Unified National Test (UNT) used in Kazakhstan since 2004 can hardly serve as the source of such information. Today it is defined as one of the forms of selective examinations to be admitted to universities.
“We have only one-fifth part of tests [in UNT] designed to check functional literacy,” wrote Assem Issabekova, researcher-analyst in education.
After surveying almost 300 Kazakhstan teachers, she found out that 92 per cent of teachers were sure that students needed remembering and overlearning skills to pass the UNT.
“Get prepared” for PISA
“Orleu” National Career Development Centre is one of the state organisations that consistently works with teachers to prepare for PISA. Teachers are generally taught to develop and apply tasks regularly in classes to develop functional literacy in students “for high indicators in international studies”, according to one of the press releases.
In October 2021, the Centre for Educational Programmes (a part of the system “Nazarbayev Intellectual Schools”), in turn, offered four guides on development of mathematical, scientific and reading literacy to Kazakhstan teachers. The motivation has been clear: “to effectively prepare students to PISA-2022”.
Plans on preparation to PISA are being developed locally, as well. Thus, one of Pavlodar schools published on its website not only a list of necessary events, but also a list of teachers in charge. From the document: “carry out an ongoing workshop” for teachers, “hold teacher-parent meeting” in grades 8-9, psychological skills trainings and “experimental tests based on international studies’ programmes” for students.
The striving for getting prepared for PISA, according to Imangaliev, is a “big self-deception” comparable to the attempts of unfair coaches to ensure their athletes win at the Olympic Games with doping.
This practice, according to the speaker, is “not widely spread, yet strongly criticised” in the world. He referred to the ambiguous Chinese experience:
“They have improved education in advance of only those regions and provinces that were initially known to take part in the assessment. And they reached their goal at the cost of worsened education in other regions that did not take part in the assessment.”
10 billion tenge for the “upgrade”
In Kazakhstan, the course toward preparation for PISA 2022 was informally announced at the end of 2019 after a broad statement of Aimagambetov. When commenting on the poor results of the last round of the assessment, he emphasised:
“Participants of PISA 2018 were students who did not have access to the upgraded educational content. We’ll see the effect of the upgraded curricula only after PISA 2021 [the cycle was postponed for a year due to the pandemic – author’s note].”
The upgraded secondary education content shapes the system, where priority is given not to the total amount of knowledge, but to the skills of a student to apply it in practice. Particular improvements were to be, for example, intersubject integration, inquiry-based learning, trilingualism, criteria-based assessment system. Kazakhstan secondary schools were planned to use the experience of “Nazarbayev Intellectual Schools” and to apply it.
The “upgrade” was introduced in the republic since 2016, last year the gradual process ended. The state programme for development of education and science for 2016-2019 meant implementation of 26 events as part of the “upgrade”, with total budget 9.8 billion tenge (21.94 million dollars).
“Heavy expenses must be justified somehow, yet Kazakhstan has no objective assessment of school knowledge quality,” Imangaliev explained the determination of the relevant ministry for “good” results in PISA 2022.
The role of context
The pedagogical excellence of teachers and nature of particular tasks in math or literature classes alone cannot provide Kazakhstan with “high indicators” in PISA, despite their importance. This is evident, in particular, from the analysis of contextual factors affecting educational achievements of Kazakhstan students. To collect data about these factors, every PISA participant fills in the questionnaire consisting of over 70 questions about their family and home background, language of instruction, attitude to reading, perception of life, school and learning time.
Analysts of the “Beles Centre for Analysis and Strategy”, having studied the performance of Kazakhstan students in PISA 2018, identified that the context (particularly, school location, language of instruction and socioeconomic status) affected up to 6.5 per cent of points of a student in mathematical, 24 per cent in reading and 18.7 per cent in scientific literacy.
“In terms of the score, a student coming from a non-needy family living in a city with Russian language of instruction gets up to 56 points on average in mathematics, up to 100 points on average in reading, and up to 82 points on average in natural science,” researchers explained the analysis.
Two more international studies, PIAAC and ICILS, came in their area of concern. Analysis of their assessments, together with PISA, showed that one of the main influential indicators in all three cases was the number of books at home. It influences the development of students’ skills in 3 to 6 per cent.
The above things do not hinder the relevant ministry from projecting future results of PISA 2022 for an average student – 430 points in mathematics, 392 points in reading, and 402 points in natural science. These strategic indicators have been outlined in the national project “Quality education “Educated nation”. If these indicators are not reached, the public will have a good reason for doubting both the quality of domestic education, and effectiveness of the current government.
Peculiarities of the system
According to social surveys, just over a half of Kazakhstanis are more or less satisfied with the level of secondary education in the republic. Experts name only few reasons of this situation.
First, domestic schools have lately been trying to cope with the increasing load, according to Saule Alieva, researcher of PaperLab and one of the authors of the comprehensive research of educational inequality in Central Asian states. Given the relatively stable number of general education schools, the school enrolment is constantly increasing. From 2010 to 2020, Kazakhstan had the decrease in the number of schools by 399 schools, whereas the number of students increased by 929 thousand students.
Second, the general educational curriculum in Kazakhstan has tended towards functional literacy just recently, said Alma Iglikova, deputy principal of the Sh.Ualikhanov private school.
“The academic programme does have such objectives. But we need to consider the implementation of such objectives by teachers, who have completed secondary schools focused on fundamental knowledge instead of “success skills.” Therefore, correct implementation is significant here,” the speaker said.
Moreover, Iglikova focused on the authoritative role of parents in improving the quality of education of their child (however, they do not have an opportunity to influence the system directly).
“One of the underestimated opportunities of influence on quality improvement by parents is helping the child in self-organisation and self-direction,” she said. “Consistency and succession in education give a positive improvement despite the child’s academic skills. Even a student with “non-mathematical mind” can achieve great results with system and consistent work.”
What (not) to expect from PISA 2022
Results of the study will be available in December 2023. No expert respondents can predict the exact figures of PISA-2022, yet they are not hasty in super-positive assessments.
“Transition to the updated content of education is an ambitious reform. I rely on it, Yet I am not sure if results will be available soon,” Alieva said.
The Information and analysis centre agreed with her.
“We’ll see the fullest effect from the upgraded content from the participants of PISA 2025,” according to the coordinator of the study, Rizagul Syzdykbaeva, of the Information and Analysis Centre.
The pandemic and the forced transition to online education can also affect the study results for Kazakhstan. Or they will become an easy excuse for poor performance, Nurlan Imangaliev said. In 2022, a new module “Global Crisis” about educational process organisation, complications and experience of distant study during closure of schools because of Covid-19.
The problem, according to him, is that Kazakhstan still has not decided how to assess the damage caused by months of distant study:
“Does it mean to catch up hours that were lost or catch up skills that were never mastered? No one has ever answered this question openly.”