Excellence is extra-ordinary

Thirty years of focus on excellence in Dutch science policy

How the science system works

Report

Downloads

ANP-16404734 Excellent is niet gewoon.jpg

In this report we analyze thirty years of excellence policy and its effects on research and researchers. We also give options to do things differently.

Authors

For thirty years, fostering excellence in science has been a key objective of Dutch science policy. Over the years, ever more policy instruments have been added, in the form of grants and prizes.

Today, excellence has become an essential core value – after all, no one can object to excellence. At the same time, many people see this focus on excellence as the cause of a development within science that they do object to:

  • increased competition;
  • the pressure to publish; and
  • the ever greater discrepancies in the appropriation of research funding among research groups.

These developments form the background for this report.

Purpose, questions and approach

In this report, the Rathenau Instituut describes the effects of a set of policy instruments that encourage excellence in science, such as the Talent Scheme (with the Veni, Vidi and Vici Grants) and the ERC Grants from the European Research Council. The aims of this report are to give an insight into the effects of the excellence policy on research practice, and to offer perspectives for adjusting that policy.

We address two questions. The first relates to the way in which excellence in research is fostered: does it deliver the intended result, and is it effective? The second considers the various tasks of the universities: does this one-sided focus on excellent research upset the balance at universities?

The results and conclusions of this report are based on an analysis of quantitative and qualitative data. We have used the following sources:

  • policy texts and political debates;
  • figures collected systematically by the research funding bodies;
  • more than fifty interviews with researchers;
  • discussions with other stakeholders;
  • and scientific literature and previously published reports.

Effective excellence policy: big impact with limited budget

This analysis shows that the policy for scientific excellence has been effective, over the past thirty years. Despite their limited budget, the grants and prizes exercise a strong guiding influence on the science system and the performance of research groups. This impact is in line with the objectives of the excellence policy. The policy has resulted in:

  • the selection of a relatively small number of researchers;
  • the concentration of resources within this group; and
  • differentiation between research groups with ample funding and those with a restricted budget.

It has emerged that excellence funding gives researchers and research groups a relatively considerable degree of freedom to set their own course and lay down their own research lines. Furthermore, success breeds success, thereby creating opportunities for acquiring new excellence funding and ensuring continuity.

Unintended effects of the excellence policy

In addition to the intended effects, the policy has revealed three negative consequences:

  1. The system of allocation of research budgets on a competitive basis is becoming increasingly costly and time-consuming. The quality differences between many grant applications are so minimal that coincidence and luck are becoming important factors in the awarding of grants. Furthermore, the reputation of a grant applicant starts to play a role: a previous winner has a greater likelihood of receiving a new grant or prize. Together this raises the question of whether the selection process is still effective and efficient, and whether it truly selects on the basis of innovation and talent.
  2. The focus on excellent research leads to less attention and appreciation for the other core tasks of the university: education and knowledge exchange. It is also at the expense of other valuable research that fails to satisfy the dominant ideas on what is excellent, and as such also fails to satisfy the criteria of the excellence programmes. As a result, for example, individual, interdisciplinary, interactive and non-mainstream research easily becomes disadvantaged.
  3. Research groups without excellence funding feel constant pressure to acquire precisely that form of funding, which in turn only further increases their pressure of work. This not only relates to financial necessity, but also the status furnished by such grants and prizes.

Options for adjustment of the excellence policy

Where should we go from here?

  1. One possibility is to continue on the present course.
  2. One alternative option is to (once again) reserve the use of excellence for the truly exceptional; for developments that stand out above all others.
  3. Another possibility would be to apply the term to first-class education, remarkable forms of cooperation, exceptional valorisation activities, etc.

All options have advantages and disadvantages. We explain them in the Recommendations tab.

Preferred citation:
Scholten, W., L. van Drooge and P. Diederen (2018). Excellence is extra-ordinary – Thirty years of focus on excellence in Dutch science policy. The Hague: Rathenau Instituut

This report call upon all stakeholders – researchers, administrators, funders and policymakers – to reconsider the meaning of excellence in today’s academic context, and how it can best be fostered. The challenge is to arrive at a vision of a balanced relationship between the various tasks of the university, and then translate that vision into appropriate forms of funding.

There are 3 options:

  1. One possibility is to continue on the present course. In that case, disadvantages such as constant pressure from competition and the lower status of education and knowledge exchange are simply the price that has to be accepted in return for a leading position in the academic rankings.

    It is highly questionable whether this option is in fact tenable. The unintended side effects are broadly recognised as clear obstacles, and represent sound reasons for investigating possible alternatives.
     
  2. One alternative option is to no longer employ excellence as a general standard but to (once again) reserve its use for the truly exceptional; for developments that stand out above all others. This would require universities to concentrate on ‘sound scientific research’: sound research is good while excellent research is truly exceptional – and as such not the standard that everyone should be expected to meet.

    It would also call upon the funding bodies to give excellence programmes a truly exclusive character, for example by considerably reducing the budget for the Talent Scheme, thereby freeing up funds for spending elsewhere.
     
  3. Another possibility would be to introduce differentiation into the term excellence rather than restricting its scope. Instead of applying the designation ‘excellent’ exclusively to excellent fundamental research, which is published in leading scientific journals, making sure it is also applied to first-class education, remarkable forms of cooperation, exceptional valorisation activities, etc. After all, a university that meets a variety of societal needs can certainly stand out in more than one direction.

    Introducing differentiation to the term excellence could help re-establish the balance between the tasks of the universities. Such an approach does however engender the risk that a number of bottlenecks, such as the high costs of budget apportionment and the emphasis of performance measurement, will also start to emerge with regard to the other tasks of the universities.

Frequently asked questions

Excellence is a complicated term and many people, including people from the academic world, find it difficult to come up with a single clear definition.

In everyday usage, the word excellent means little more than ‘very good indeed’. In the academic world, however, it has acquired a very specific meaning as a result of the way it is implemented and applied. This is confirmed in our study. 

The meaning is determined by a combination of university culture and science policy, and consists of the following 7 points:

  1. Excellent is a term that mainly refers to research and researchers.
    In principle, education, knowledge exchange and management can also be deemed excellent, as can the people who provide them, but in practice they are rarely referred to by the term.
     
  2. Excellence is associated with fundamental and innovative research, with the pushing of boundaries and achieving breakthroughs.
    Applied research or replication research can also be excellent, but in practice the term is far less often associated with these forms of research.
     
  3. Research and researchers who stand out above all others are excellent.
    For that reason, excellence can only be determined by comparison – and therefore via competition (for research funding, for publications). A focus on excellence is intrinsically linked to competition in the academic world.
     
  4. Competition here mainly means competition between individual researchers.
    At issue is the ground-breaking research programme or research project that helps scientists achieve major steps forward. It is the talented individual researcher who rises above ground level.
     
  5. Competition above all refers to national and European competition for research funding.
    At faculty and institution level, there are almost no resources available for investing in research and researchers.
     
  6. To compare research and researchers, excellence must be observable and measurable.
    Otherwise there is no competition. A focus on excellence in the academic world has therefore led to huge attention for the registration and measurement of performance (both delivered and still to be delivered).
     
  7. To be able to measure excellence, uniform yardsticks are required.
    These are essential for comparing ‘apples with oranges’ – after all, no two studies are exactly the same. More or less uniform yardsticks have been found in publications and citation indexes for performance delivered and in checklists and points systems for the assessment of research proposals.

The funding instruments aimed at fostering excellence were introduced to allow excellent researchers to carry out free and innovative research work. By focusing on topflight scientists, policymakers in the Netherlands and many other European countries aim to improve the effectiveness and efficiency of scientific research. In this way, it is hoped that Dutch science will keep up with leading science around the world.

Rewarding a small group of top scientists inevitably means that other researchers receive less funding. Fostering scientific excellence is based on five assumptions:

  1. The first assumption is that leeway for free and unfettered research is of considerable value to society, but that value cannot be estimated in advance.
    The value lies not only in satisfying scientific curiosity but also in the possible application of knowledge. It is often not possible to predict in advance which fundamental research will eventually lead to useful applications or insights usable in practice. Many technological developments are the result of scientific research that was initiated for completely different reasons and with completely different expectations. In that sense, it is meaningful to operate a series of policy instruments within science policy that focus exclusively on excellence rather than on content.
     
  2. A second assumption is that greater freedom for the best researchers leads to more ground-breaking research.
    The best researchers stand out not only due to the excellent mastery of their specialist field but also their creativity and originality. They are the best positioned to determine what needs to be done in order to push boundaries and advance their discipline. It is therefore essential that they be offered sufficient leeway to obtain funding for research proposals they have developed themselves. In that way, the likelihood of achieving original radical advances is greater than if they are required to respond to programme-based calls in which research is required to answer a whole series of pre-formulated questions. The instruments discussed above deliver that leeway. They also encourage the taking of risks, which is less likely with other forms of funding. Individual prizes are better at creating this freedom than grants.
     
  3. A third assumption is that encouraging top scientists – and in that way promoting vertical differentiation – has a positive effect on the environment in which these eminent scientists operate.
    People who perform at an excellent level inspire their environment and attract other excellent researchers. The development of an excellent research group and an excellent research institute often starts with an exceptional, individual scientist. In other words, excellent research has ‘spill-over effects’ (Cremonini & Jongbloed 2017). The implication is that dynamic developments can be achieved more effectively by supporting excellent researchers than by supporting science across the board or encouraging areas of science that lag behind.
     
  4. A fourth assumption is that it is possible to distinguish between a proposal for excellent research and a proposal for non-excellent research.
    Excellence can be defined, recognised and assessed. Focusing on excellence implicitly contains the assumption that it is possible to select the best proposals and best researchers in competition. 
     
  5. A fifth assumption is that organising a competition is the most suitable means of identifying the best performing and most promising researchers.
    As a consequence, government funding for scientific research automatically ends up with the most excellent research. This works best if the instruments focused on excellence are not based on content. Only if scientists from all fields are able to compete with one another can those who stand out above the rest be identified.

However, questions can be raised regarding each of these assumptions.

No one opposes top science or topflight science. It can lead to scientific breakthroughs and may be used to solve the problems of society. 

Nonetheless, the effects of the focus on excellence have been the subject of debate for some time. They have been linked to negative developments in the academic world, such as:

  • increased application pressure
  • high pressure of work
  • performance pressure
  • publication pressure
  • increased competition
  • inequality or even unfairness in the appropriation of public funding for scientific research 

As a result, although they recognise the importance of striving for excellence, many policymakers, administrators and researchers remain critical of the specific way in which it is implemented and its effects.

In many rankings, Dutch science performs extremely well. It is however difficult to prove whether this is the result of the funding of scientific excellence.

We can show that the combined use of the policy instruments referred to above does result in:

  • the selection of a relatively small number of researchers;
  • the concentration of funding among these researchers; and
  • differentiation between research groups with plentiful and those with limited funding.

The funding instruments establish the conditions that encourage excellent-quality research such as:

  • autonomy for the research leader;
  • opportunities for developing research projects with a long-term perspective;
  • financial freedom to invest in expertise and facilities.

These outcomes are in line with the thinking behind the policy. However there are also three negative effects of focusing on scientific excellence:

  • Research groups without excellence funding experience constant pressure to obtain this form of funding, which in turn increases the pressure of work. This relates not only to financial necessity but also the status acquired through grants and prizes.
  • The system for allocating research budget through competition is becoming increasingly costly and time-consuming. The quality differences between many applications are so minimal that coincidence and luck have become important factors in the awarding of grants. Moreover, the reputation of an applicant also starts to play a role: recipients of prior funding have a better chance of being awarded a grant or prize (the so-called Matthew effect). All these elements raise the question whether the selection process is still effective and efficient, and whether it selects on the basis of innovation and talent.
  • The focus on excellent research leads to less attention and recognition for the other core tasks of universities: education and knowledge exchange. It is also detrimental to valuable research that fails to satisfy the dominant ideas on what excellence is and as a consequence does not satisfy the criteria for excellence programmes. Individual, interdisciplinary, interactive and non-mainstream research are often easily disadvantaged.

The answer to that question depends on how you define and measure excellence. Furthermore, huge discrepancies can arise between for example universities and domains.

Worldwide rankings are often used to define the quality of science at a university or in a particular country. Different rankings measure quality in different ways.

The most widely used method for measuring excellence is by counting numbers of scientific articles, and how often those articles are used in other articles. The Netherlands and Dutch universities have achieved high to very high scores on these rankings for years. In addition, the differences between the universities in the Netherlands are far smaller than in many other countries.

There are two important questions relating to the use of rankings:

  1. Counting numbers of articles does not work equally well for all domains, because in certain domains, for example, researchers particularly write books.
  2. Researchers also do other things besides writing articles in scientific journals, and those activities often remain unmeasured.

In the majority of research grants, the quality of the research proposal and of the researchers is a criterion employed by the selection committee in assessing the proposal. In other words, high quality is always essential with regard to research funding.

In the case of a number of grants, quality is practically the only selection criterion. These are referred to as excellence grants.

This may relate to grants for which a researcher is required to submit a proposal, or to prizes for which a person is nominated on the basis of historical performance. Here we will list only those grants that are open to all domains.

These are the Dutch excellence grants

The NWO Talent Scheme was established in 2000 and since 2002 has consisted of the following grants:

  • Veni Grants
    Veni Grants are only available to researchers who obtained their PhD not more than three years ago. The maximum amount of the grant is 250,000 euros and the aim is to enable young researchers to further develop their ideas. The person to whom the grant is awarded (the laureate) can take three years to spend this funding.
     
  • Vidi Grants
    The Vidi Grant is available to researchers who obtained their PhD up to eight years ago. The maximum amount is 800,000 euros. The grant is intended to help the winner set up their own ground-breaking research programme, and to appoint a number of PhD candidates or postdocs. The budget must be spent within five years.
     
  • Vici Grants
    More experienced researchers can apply for a Vici Grant up to fifteen years after obtaining their PhD. The grant enables researchers to set up or expand their own research group, thereby giving the research programme a more permanent footing at a Dutch research institution. The maximum grant amount is 1.5 million euros. This budget must be spent within five years.

    The predecessor to the Veni, Vidi and Vici Grants, introduced by the science funding body NWO in 1989, was the PIONIER programme (Individual Grants for Research Groups with New Ideas for Excellent Research).
     
  • Gravitation programme grants (since 2013)
    These grants from the Ministry of Education, Culture and Science (OCW) are meant for consortia of researchers that bring together the best researchers in a particular domain. They number among the best in the world in their field or show the potential to join that top group. The grants – 50 million euros are available for appropriation each year – are awarded for a maximum period of ten years.
     

The predecessors to the Gravitation programme were the Top Research Schools (1998-2012), which themselves emerged from the grants for Research Schools that were introduced in 1992.

These are the European excellence instruments

  • The Starting Grant (since 2007)
    This European Research Council (ERC) grant is intended for young researchers (2 to 7 years following their PhD) to help them independently set up their research programme and research group. The maximum amount of the grant is 1.5 million euros over a five-year period.
     
  • The Consolidator Grant (since 2013)
    This ERC grant is aimed at further establishing the independent career of a researcher and the position of a young research group. The applicant must have 7 to 12 years post-PhD experience. The maximum budget is 2 million euros, also to be spent over a five-year period.
     
  • The Advanced Grant (since 2008)
    This ERC grant is intended for excellent and eminent researchers who have demonstrated the transformative and ground-breaking nature of their research. The maximum amount of this grant is 2.5 million euros, over a five-year period.
  • The Synergy Grant (since 2012)
    This ERC grant funds small consortia of between two and four chief researchers and their research groups. The aim of the grant is to offer opportunities to researchers who wish to answer a particular research question jointly, on the basis of complementary qualities. The maximum amount of the grant is 15 million euros to be spent over a period of six years.

These are the excellence prizes

  • The Spinoza Prizes (since 1995)
    These prizes deliver an additional boost to further topflight research and represent an annual distinction for between two and four excellent researchers who rank as the absolute best in their field, according to international standards. The Spinoza Prize is currently worth 2.5 million euros. The prize is only open to researchers who have not only established a superb scientific career but who are also in a position to initiate new research.

    Between 2002 and 2016 the KNAW awarded the annual Academy Professors Prize to at least two excellent professors who through their career had demonstrated their position at the absolute top of their field. It was a lifetime prize that enabled the laureate to focus entirely on research, for a period of five years. The KNAW also expected the employer of Academy Professors to use the budget thus released to appoint talented group leaders and professors.