Consequences of the EGE in Russia

Special Issue:

Consequences of high-stakes assessment from an international perspective

Guest Editor:

Hoi Suen

KEDI Journal of Educational Policy

Volume 3, Number 1, 2006

Consequences of high-stakes assessment from an

international perspective

Contents

Editorial

1 Hyung Yeel Koh

Guest Editor's Preface

3 Social consequences of high-stakes assessment: An international

perspective

Hoi K. Suen

Articles

7 Consequences of college entrance exams in China and the reform

challenges Haifeng Liu and Qiong Wu

23 Highstakes testing with preschool children: Violation of professional standards

for evidence-based practice in early childhood intervention Stephen J.

Bagnato and Hsiang Yeh Ho

45 Testing's influence on the arts: Some unexpected findings from Virginia Mindy L.

Kornhaber, Jacob Mishook. Maeghan P. Edwards, and Takako Nomi

69 Highstakes testing and its impact on students and schools in Hong Kong: What

we have learned from the PISA studies

Esther Sui-chu Ho

89 Consequences of the EGE in Russia

Vadim Avanesov

101 Consequences of high-stakes testing on the family and schools in Japan

Yoshitaka Saito

113 Psychometric paradox of very high-stakes assessment and solutions Hoi K.

Suen and Qiong Wu

Book Review

131 Private tutoring: The shadow education system

Michael J. Seth

KJEP 3:1 (2006), pp. 1-2

Editorial

One of the hot topics in the contemporary educational discourse is high-stakes testing,

especially in terms of what it can do for student learning and school accountability. The

Editorial Board of the KEDI Journal of Educational Policy is pleased to have an

opportunity to address this topic from an international and comparative perspective,

highlighting why and how high-stakes testing is to be developed and administered in

differing historical, cultural, and political contexts.

Dr. Hoi K. Suen of Pennsylvania State University has kindly agreed to guest edit this

issue. On behalf of the Editorial Board, I express sincere gratitude. The seven articles

present interesting viewpoints and informative contexts of high-stakes testing - positive and

negative arguments on its effects, detailed descriptions of how it has evolved in various

national contexts, as well as its current trends and future prospects.

Whether or not high-stakes testing improves student achievement and teacher instruction

differs greatly depending upon the situation and the expected goals of the society

concerned. As we learn from the articles, high-stakes testing has long been an integral part

of the education system in many Asian countries while other countries such as the U.S. and

Russia have just begun to seriously try out the large-scale tests as part of their systemic

reform measures.

^*KECll KEDI Journal of Educational Policy- ISSN 1739-4341 -

(F1 Korean Educational Development Institute 2006, Electronic version: http://eng.kedi.re.kr

Editorial

I hope that this special issue will enable us to broaden and deepen our knowledge and

thinking on high-stakes testing and related issues. My heartfelt thanks are always rendered

to KEDl's publication team for the hard work.

Hyung Yeel Koh, Ph.D.

Editor-in Chief of KEDI Journal of Educational Policy President of

Korean Educational Development Institute

KJEP 3:1 (2006), pp. 89-99

Consequences of the EGE in Russia

Vadim Avanesov

Educational Measurement, Russia

Abstract

The essence and consequences of the high-stakes Common State Examination (EGE) in

Russia are analyzed. The system began as an anti-corruption effort, but then was changed to

being a means to the commercialization of education. It has neither improved education, nor

overcome corruption. The best way to resolve the problems is to privatize this examination

to a non-governmental national testing center.

Keywords: governmental examination, test commercialization, unreliability, negative

educational impact

£ISSEl! KEDZ Journal of Educatiotial Policy- ISSN 1739-4341 -

© Koreati Educational Development Institute 2006, Electronic version: http://eng.kedi.re.kr

Vadim Avanesov

At the end of 1999, at the very height of the transfer of Russia from socialism to

capitalism, a group was created within the Center for Strategic Developments. This group,

whose composition has never been revealed, proposed to the government of Russia to

replace final examinations in secondary schools and entrance examinations at universities

with one overall state (governmental) examination named the Common State Examination

???????????????????????????????), or in abbreviation the EGE (???). The first word

?????? means one common, instead of two examinations - one at the end of secondary

school, and a second at the entrance to universities. The second word may be translated as State in the sense of governmental because it is organized, conducted, and paid by the government of the Russian Federation via the Ministry of Science and Education. The third word means examination in English.

According Item 3 in Article 43.3 of the Russian Constitution, "Everyone has the right to

receive, free of charge and on a competitive basis, higher education in a state or municipal

educational institution or enterprise." However, in the project, only 5% of students who

score above a certain point on the EGE were provided with a free higher education. All

others had to pay. Specifically, five levels of payment were established, dependent on

scores on the EGE. By this mechanism, the Russian government has tied the amount of

tuition and fee for higher education to EGE scores. This scheme was named

(Governmental Personalized Financial Obligations or abbreviated as GIFO. The main idea is

to introduce EGE and GIFO together as a means to determine payment for education. Consequently,

the EGE/GIFO has brought about a new standard, which is not in Constitution; namely, the lower the EGE

scores, the higher are the payments among those who are admitted into a university on

competitive basis. Thus, the EGE was planned as a type of "high-stakes examination," the

results of which would influence not only the fate of secondary school graduates and

university entrants, but also would allow the government to require payments for almost

95% of the admitted students in higher education.

Problems arose from the moment the governmental decision was made to introduce this

new unified examination as compulsory for all secondary school graduates and university

entrants. The impetus for this examination can be traced to governmental attempts against

corruption, which has noticeably become widespread in Russia until and after the change of

the social system in 1991 (e.g., Avanesov, 1998). The main forms of corruption had been

widespread bribery and nepotism.

The EGE was thus initiated as a reform movement but has by now gradually converted

into being the cause of a chasm between political authorities and the citizenry; as well as a

point of contention between supporters and opponents of the EGE. Numerous publications

in the press have voiced support for and criticism against EGE, including sarcastic articles

such as one entitled "the show must go on" have appeared. Additionally, ordinary citizens,

civil

Vadim Avanesov

the level of corruption in the educational sphere. Struggle against corruption was and is

acutely necessary for Russia.

Although it is true that college entrance testing is often considered a means to combat

corruption or nepotism in many countries (e.g., see the article by Liu & Wu about China in

this special issue), the former Russian Ministry of Education, which was charged with the

implementation of the EGE system, actually refused to use it to prevent corruption. When

asked about the anti-corruption function of the EGE, the Minister of Education responded

that the aim of the EGE was "completely not for that... I can officially say that combating

corruption is generally the work of other departments, not that of the Ministry of

Education." He further stated that it would have been impossible to combat corruption in

education because the field of education is occupied by intelligent people who can devise

clever ways to make corruption look legitimate. As an example, he described the story of

the university rector who, instead of blatantly asking for a bribe of 5,000 dollars from his

friend to guarantee the admission of the friend's daughter to the university, the rector stated

to his friend, "Let's bet 5,000 dollars that your daughter will be admitted on her own right

without bribery" (Company for the Development of Public Relations, 2003).

Multiple functions

Aside from the more lofty political goals of social equality and anti-corruption, the EGE

has a number of operational functional goals. Bakker (2005) identified at least four such

functions: 1) as the final examination for high schools; 2) as the entrance examination to

universities; 3) for the awarding of grants for university education (i.e., GIFO); and 4) as a

general measure of educational achievements in the Russian Federation. The third function

of awarding grants for university education is due to the GIFO portion of the EGE/GIFO

pair. Specifically, GIFOs are state nominal financial obligations. This system assumed an

objective distribution of the means for higher education. The categories of GIFO to which a

student belongs will depend on his/her EGE score. The highest scores will ensure a free

university education. But for those who score lower, the higher will be the payment. The

EGE score was planned to be the basis for the allocation of funds for higher education.

Immediate consequences of the EGE-GIFO link

The linking of EGE scores to the GIFO system of tuition payment almost immediately

led to some observable economic and political consequences.

Consequences of the EGE in Russia

Economic consequences

It should be noted here that the function of the EGE as a basis for awarding grants, as was

identified by Bakker (2005), is true only under a system that no one is entitled to a free

higher education and therefore must compete for financial grant awards through the

EGE/GIFO system. However, given the Russian constitutional guarantee of a free higher

education for its citizens, essentially, the federal government has removed itself through the

EGE/GIFO system from the financial responsibilities for higher education for the bulk of

young people. That is, the effect of the system is directly contrary to both the letters and the

intents of the Russian Constitution.

Indirectly, the EGE/GIFO has become a first step toward the privatization of higher

education. In July 2000, the federal government issued Document ? 1072 in which the

status of universities changed from that of "State Educational Establishments" to a new

status of "Educational Organizations" (Ministry of Education and Science of the Russian

Federation, 2000).

This opened the way to the rapid privatization of universities. This was implemented in

spite of objections from the Russian State Duma and from the Congress of Rectors of

Russian Universities.

Ironically, and unexpectedly, instead of minimizing its financial responsibilities for

higher education, the EGE/GIFO has actually led to a heavier financial burden for the

government. First, as a result of the EGE/GIFO system, universities are given a new right to

set their own tuition. Thus, elite universities would establish higher tuitions while others

would set lower prices. Second, since universities were interested in obtaining high sums of

GIFO funds from the government on the basis of EGE scores, these scores have began

artificially and irrepressibly risen both under the pressure of corruption and financial self

interests. As a result of both over-charged tuition and inflated EGE scores, the Government

of Russia was forced to pay more than was originally intended. While the cost for higher

education has risen due to the use of the EGE, some of the smaller local universities will

become economically unviable and will be closed.

Political consequences

Although universities have gained their abilities to set their own tuitions, they have lost

their right to determine their own entrance examinations and criteria. The EGE has thus

undermined the autonomy of universities established by law. Matters such as educational

standards, certification and accreditation of educational establishments are supposed to be

within the purview of educational institutions only. Through the EGE, the federal

government has imposed their will on the educational establishments of the country. Yet,

entrance in universities is a complex scientific problem of educational selection on the basis

of the ability of an applicant in mastering the contents of the

Vadim Avanesov

specific educational program. It cannot be solved by a simple and sweeping EGE/GIFO.

Rather, it requires the creation of independent regional and intra-universities testing centers

and by the development of sound systems of personnel selection. Yet, educational

institutions have lost their abilities to make such determinations.

Impact on corruption

Ironically, even though one of the official goals of the EGE43IFO system is to fight

against corruption, as a result of the financial elements of the GIFO portion of the system,

corruption has increased instead. The corruption has been further exacerbated by the lack of

transparency on the part of the Government of the Russian Federation around the results of

the system. Reports of illegal manipulations of EGE test results are widespread.

Kuz'minov, Rector of the College of Economics in Moscow, estimates that about 10% of

EGE scores today are the results of corruption (Lemoutkina, 2005).

The EGE has led to a new turn of corruption surrounding the testing and funding system.

Because of the absence of genuine public control and accountability, corruption has reduced

Russian education to that of a second-class status. In 2005, the level of corruption in Russia

has placed it from the 90th place in the previous year to the 126' places in terms of clean

government, making Russia on par with such developing nations as Niger and Sierra Leone

(The United Nations, 2004).

Impact on education

It is difficult to evaluate the success of the EGE as only limited data are available.

Judging from the limited statistics, the implementation of the EGE is going well. The extent

of implementation is confirmed by official statements and publications. According to the

Russian Ministry of Education and Science, in 2005, 850,000 high school graduates from

78 regions took the EGE, which were 30,000 more than it had been in 2004. The

implementation will expand to two additional regions in 2006, making the total number of

examinees around 1 million (Lemoutkina, 2005).

At present more than 78 regions of the Russian Federation presented the "matched

solutions" from the participation in the experiment in 2006-2008. Also, the number of

regions participating in all subject areas has steadily increased, from 4 in 2003, to 8 in 2004

to 14 in 2005. In 2003, 19 regions participated in 6-11 subjects. This increased to 35 regions

in 2004 and to 51 regions in 2005. (Bolotov, 2004). There is clearly a steady increase in the

number of regions participating in the EGE. However, this increase can be at least parti-

Consequences of the EGE in Russia

ally attributed to the financial incentive for participation. Funds for the administration of the

EGE are allocated in such a way that participating regional authorities are provided with

half of the funds.

Consequences on the quality of education

These quantitative increases beg the question as to whether there has been an

improvement in the quality of education. Due to the lack of program transparency,

information is difficult to obtain. Available data are often untrustworthy. For example,

according to a report by the Liberty radio station, at one of the territories of Russia in 2005,

the EGE scores improved a miraculous 2,800 percent (Kostinsky, 2005).

In the absence of reliable statistics, the general public opinion is that the quality of

Russian education has deteriorated. From 5 to 10% of school-age children are not attending

schools today. Those who attend schools are reading less. School compositions are largely

replaced by rote learning of model writings. The Minister of Science and Education,

Fursenko(2005) publicly admitted the worsening of the quality of education in Russia.

However, he blamed the problem on the low wages of teachers, reduction in interests in

learning, and also the absence among students of "a clear understanding of how the quality

of knowledge and education can actually change their lives." Test coaching classes and

private paid test coaches have begun to appear everywhere. These coaching classes focus on

only test-taking skills without helping students to advance their knowledge.

The declared aims - an increase in accessibility and quality of education -are not achieved

by means of the EGE. Results of independent research indicate that the effect of the

EGEG1IFO voucher experiment increases inequality in access to higher education, and that

it is difficult to see any improvements of efficiency. Russia is characterized by a large

inequity between rural and urban areas. Market governance through vouchers introduced in

such context seems doomed to increase inequality (Kleshchukova, 2005).

The other aim of combating corruption has also failed. Instead, corruption has grown.

The payment for education reached an unprecedented level. Now, 57% of college students

must pay for their education. Although the official goals for the EGE are for various

reforms, the evidence suggests that the true aim is the commercialization of education in

Russia, in spite of the fact that the income of the general population is low and can ill-afford

privatized education and the general public is against such an educational policy.

The Rector of Moscow State University, Sadovnichiy (2001) stated, "As I have

repeatedly said, the gap between higher education and general education in secondary

schools has reached a catastrophic level. Many specialists have projected that this gap will

lead to the lowering of university standards to the levels of secondary schools." This gap in

knowledge between secondary school

Vadim Avanesov

graduates and college entrance requirements has further hampered the possibility of

establishing an EGE appropriate for college entrance.

Consequences on the equality of educational opportunities

As a democratically oriented country, Russia has the obligations to ensure equal

opportunities and not to erect barriers for its citizens to obtain education. The EGE is

essentially a government-sponsored barrier to limit educational opportunities to its citizens.

With the EGE, the cutoff score for a given university can be easily manipulated by the

university to maximize the financial responsibilities of students. This unavoidably reduces

the number of citizens who can attend a university. Hence, contrary to its goal of increasing

educational opportunities, the EGE-GIFO system has deprived individuals from low-income

backgrounds the opportunity for higher education. This has polarized education into those

for the elite and those for the mass (Avanesov, 2000) Mass education begins to degrade and

universities have been turned into diploma mills.

To illustrate, the Republic of Saha (Yakutiya) was among the first to participate in the

EGE social experiment. As a result, in 2003, only 25% of the freshmen in their universities

were able to enroll without payment. The remaining 75% were obliged to pay due to their

EGE-GIFO results. It is clear that, instead of ensuring the equality and accessibility of

higher education to Russians, the system has limited the opportunities to only those who can

pay. And this is in spite of the insistence by President Putin that the basis of the national

policy is free education.

Fjodorov, the Rector of Moscow Technical University, stated that "EGE-GIFO enlarges

the paid component in education and thus, decreases accessibility. The EGE-GIFO will not

work anywhere in the world. Where it has been tried, the citizens have rejected it"

(Fiodorov, 2000).

Summary and discussion

Many independent authors, including many prominent scientists, consider the EGE a

powerful destructive force on Russian education, which had until recently, enjoyed a high

status in the world. Some officials have compared the destructive force of the EGE to that

of the momentum of an automobile, which is difficult to stop suddenly. Perturbed by

negative consequences of the EGE, the scientific community recently submitted an open

letter to President Putin to voice their protest against the system. The letter pointed out the

destructive nature of the system to Russian society and called for an end to the EGE.

They listed seven major reasons for the termination of the testing system.

Consequences of the EGE in Russia

Among these reasons against the EGE-GIFO system, the scientists identified the

privatization of education and the conversion of Russian schools to test coaching

institutions as two major sources of concern. Additionally, the lack of transparency of the

system makes mass manipulations and falsifications of results inevitable. Also, the EGE not

only cannot fight against corruption, but will redistribute and multiply it under new forms.

The letter went on to suggest several steps to be taken to improve the quality of education.

The first step is to make the results of the EGE-GIFO social experiment open and

transparent.

The question now is what can be done. First, it is necessary to fight against corruption in

any national effort, including the EGE. Corruption can be minimized if the EGE-GIFO

project were managed by professionals. Additionally, as Bakker suggested, an increase in

the transparency in the management of the EGE and the removal of vagueness of

responsibilities and procedures will help (Bakker, 2004).

It is necessary to put an end to the strategy of the commercialization of education, as a

result of which Russia has descended to the 57th place on the index of the development

human potentials (UN Report, 2004). Russia has been deprived of public education. The

commercialization through the EGE system has polarized education and the education for

the mass has deteriorated.

It is also necessary to create a structure for the civilian oversight and control of education

in general. Educational reforms without the willing cooperation among students, parents,

citizens, society in general and the government are impossible. It is necessary to understand

that education is first of all the affair of students, their parents, relatives, teachers and

professors. Therefore, effective educational reform does not come from a governmental

imposition of standards and tests, but from ensuring the quality of teachers, the adequacy of

school facilities, from adequate nutrition for the children, adequate supplies and textbooks,

from the commitment to the future of the children. Finally, it is necessary to conduct formal

studies of any damages that might have been brought on by the EGE to students.

The best way to overcome the deficiencies of EGE is to redesign it from scratch.

Foremost, the goals of the system must be clearly articulated and sufficient commitment

must be made to realize these goals. No test can serve as many purposes as the current EGE

has set out to serve. As it is and given the consequences, the EGE is not a good university

admission test (Bakker, 2004). If the goal is indeed to have a tool to fight against corruption

in education or a tool to ensure educational equity, the new appropriate tests must be

designed with these exact goals in mind without overburdening them with other objectives,

such as the commercialization of education. If the goal is to have a tool to aid college

admission decisions, another approach run by another organization other than those of the

current EGE is needed.

Human history is filled with the development of erroneous solutions to

problems that have led to unintended new problems. Many of these unintended new

problems took decades to overcome. It is reasonable to assume that the role of high-stakes

examinations will continue to grow. In Russia, the growth should be accompanied by

careful scrutiny of unintended consequences of these assessment systems. Inappropriate

assessment methods and systems can potentially lead to long-term damages to younger

generation that will take years or decades to correct.

Address for correspondence

Vadim Avanesov

Editor in Chief

Educational Measurement, Russia

Email: Адрес электронной почты защищен от спам-ботов. Для просмотра адреса в вашем браузере должен быть включен Javascript.

References

Avanesov, V. S. (1998, August 8). Admittance to universities: Painful social

problem. Independent Newspaper, p. 3. Avanesov, V. S. (2000, October 10). Two faces

of educational policy. Independent

Newspaper, p. 3. Bakker, S. (2005). Evaluation of educational achievements in the

frame of national examinations. Proceedings of the International Conference on the

Evaluation of Educational Achievements in the Frame of the National Examinations,

(p.257-263). Moscow: Uniquem Press.

Bolotov, V. (2005). The EGE as an All-Russian system of assessment of quality of

education. Proceedings of the International Conference on the Evaluation of Educational

Achievements in the frame of National Examinations, (p.12). Company for the Development

Public Relations. (2003). Wliat is the EGE for?

Retrieved March 23, 2006, from: http://www.khb.ru/articles/detail/ege.htrnl;

http://www.nakhodka.ru/educationnews.asp?id=62.

Fiodorov, I. (2002). Moscow on the way of modernization: Personnel potential as

decisive factor of effective development. Problems in the Development of Higher

Education, 31 (October), 26-27

Fursenko, A. (2005). Educational standards need to be changed. Retrieved March

23, 2006, from: http: / /www.nonews.ru/popups/rianpopup.php7varhref =/society/

200502l5/28926195.html. Kleshchukova, M. (2005). Merit-based voucher for

higher education in Russia: An analysis of the voucher experiment in the Mari Republic in

its early phase 2002-2004. Hedda Progmmm. Oslo, Sweden: Institute for Educational

Research, University of Oslo.

Kostinsky, A. (2005). About the EGE. Radio Liberty. Retrieved March 23, 2006,

from: http: / /www.svoboda.org/programs /edu /2005/edu.071305.asp. Lemoutkina, M.

(2005). EGE is postponed for 3 years. Retrieved on March 23, 2006 from:

http://www.faito.ru/news/show_news.php?subaction=showfull&id=1131825110&archive=1

138886012&template. Ministry of Education and Science of the Russian Federation. (2000).

Education in Documents ? 1072 (Governmental Document 2000 No. 18). Ministry of

Education and Science of the Russian Federation. (2004). Decision on October, 13, 2004.

Retrieved March 23, 2006, from: http: // ege.spb.ru/index.php?page=news003. Sadovnichiy

V. (2001). Universities and society: Cooperation of the universities on the boundary of the

century. Universities, Their Present and Future, March, 2-31. The United Nations (2004).

The UN report on Human Development. Moscow.