REF

About REF

The REF is the UK’s system for assessing the quality of research in UK higher education institutions. It first took place in 2014. The latest exercise was conducted in 2021.

What is the purpose of the REF?

The funding bodies’ shared policy aim for research assessment is to secure the continuation of a world-class, dynamic and responsive research base across the full academic spectrum within UK higher education. We expect that this will be achieved through the threefold purpose of the REF:

• To provide accountability for public investment in research and produce evidence of the benefits of this investment
• To provide benchmarking information and establish reputational yardsticks, for use within the HE sector and for public information
• To inform the selective allocation of funding for research. 

For more information about the REF, visit their website.

Aston Output Review Process for a Future Research Assessment Programme (REF 2028)

The generic REF criteria for assessing outputs are originality, significance and rigour:

• Originality will be understood as the extent to which the output makes an important and innovative contribution to understanding and knowledge in the field. Research outputs that demonstrate originality may do one or more of the following: produce and interpret new empirical findings or new material; engage with new and/or complex problems; develop innovative research methods, methodologies and analytical techniques; show imaginative and creative scope; provide new arguments and/or new forms of expression, formal innovations, interpretations and/or insights; collect and engage with novel types of data; and/or advance theory or the analysis of doctrine, policy or practice, and new forms of expression.
   
• Significance will be understood as the extent to which the work has influenced or has the capacity to influence, knowledge and scholarly thought or the development and understanding of policy and/or practice.
   
• Rigour will be understood as the extent to which the work demonstrates intellectual coherence and integrity, and adopts robust and appropriate concepts, analyses, sources, theories and/or methodologies.

The criteria for assessing the quality of outputs are ‘originality, significance and rigour.

Four Star - Quality that is world-leading in terms of originality, significance and rigour.

Three Star - Quality that is internationally excellent in terms of originality, significance and rigour, but which falls short of the highest standards of excellence.

Two Star - Quality that is recognised as internationally in terms of originality, significance and rigour.

One Star - Quality that is recognised nationally in terms of originality, significance and
rigour.

Unclassified - Quality that falls below the standard of nationally recognised work, Or work
which does not meet the published definition of research for the purposes
of the REF assessment.

At Aston we have added a further level of granularity to our review process. We are working to a 12-point scale in order to ensure that we have the best possible understanding of the rank order of the quality of papers for selection for the REF

Peer Review at Aston

Peer-review of outputs is conducted on an annual cycle to cover new outputs by existing staff since the last review as well as eligible outputs by new staff who have joined since the last review. The review process involves four stages:

1. Academics self-select their outputs for consideration (by 30 April annually). This requirement applies to all academic staff and independent researchers (as defined in the annual HESA return). All outputs must be in PURE and must be Open Access compliant.
2. The outputs are reviewed by at least two separate reviewers at the level of Department / Research Group (by 31 July annually). Each should allocate a score on the twelve-point scale used for REF outputs. Each should also provide a brief commentary on how the output fared against the REF criteria of originality, significance and rigour.
3. A College-level panel will moderate output scores, review scores at the equivalent of 4* as well as at 2*/3* boundary, and resolve individual scores as required under the scores entered under the 12-point scale (by 30 September annually). This will provide assurance that the assessments are robust. The panel will pay particular attention to confirmation bias in the scores. Only at this point will publications metrics, in line with Aston's Responsible Use Of Metrics Policy, be used.
4. The moderated scores will be subject to an annual Equality Impact Assessment. This, together with the score profiles, will be considered both by College and University Research Committees during each autumn.

The following Principles will be observed: 

1. For any given output, two reviewers should be identified by Deputy Deans for Research, paying due attention to considerations of diversity.
2. Reviewing should be single-blind, i.e. the reviewers are anonymous but the output itself is not anonymised.
3. The College level panel will consist of at least three senior academics at the Professorial level with significant expertise of peer review. It too, must be constituted with due regard to the considerations of diversity and the breadth of the discipline. 
4. The author of the output will have the opportunity for feedback from the Deputy Dean for Research, to include the score, as well as reviewers’ comments on the originality, significance and rigour of the output.
5. There is no appeal under academic judgement; appeals on process should be lodged with the Deputy Dean for Research by 31 October annually.

All peer-reviewers will undertake the Aston University Equality and Diversity training awareness training for data users and will in addition, receive calibration training. Only academics who have undertaken the training may conduct peer review. The training should be refreshed triennially (unless otherwise advised).

 

 

Aston's performance in REF 2021

The Research Excellence Framework (REF) 2021 has rated 79% of Aston University research as ‘world leading’ (4*) or ‘internationally excellent’ (3*).

In addition, 74% of the impact of the University’s research has been rated “very considerable” or “outstanding” for reach and significance.

The number of Aston University staff being submitted for assessment has doubled since the last REF in 2014 and, over the same period, the University has increased the proportion of its highest rated 4* publications from 21% in 2014 to 29% in 2021.

Aston University made a 100% submission of eligible staff across seven REF assessment units and has more than doubled its ‘research power’ since 2014. Research power is calculated by the overall quality of the submission (the 'grade point average') multiplied by the full-time equivalent number of researchers submitted.

Find out more.

Aston's performance in REF 2014

• 78% of research outputs were evaluated as 'world leading' or 'internationally recognised', up from 45% in RAE 2008.
• In Allied Health Professions, Dentistry, Nursing and Pharmacy, 94% of our research was judged to be World Leading or Internationally Excellent (4*/3*).
• In three Units of Assessment (Computer Science and Informatics, Electrical and Electronic Engineering, Metallurgy and Materials, and Business and Management Studies) all of our impact case studies were rated 4* or 3*.
• In the seven Units of Assessment where Aston submitted a return, 88.7% of Aston's research was found to have high levels of impact.
• In three Units of Assessment (Allied Health Professions, Dentistry, Nursing and Pharmacy; Business and Management Studies Modern Languages and Linguistics) Aston was recognised as a great place for researchers, with the vitality and sustainability of our research environment rated 4* and 3*.

How we compare

• In Area Studies we were rated in the top 3 in the UK – ahead of the universities of Oxford and Cambridge.
• We were rated in the top 5 in Allied Health Professions, Dentistry, Nursing and Pharmacy, ahead of most of the Russell Group.
• Aston Business School was rated 17th out of 101 institutions in terms of 3* and 4* impact.
• We rank in the top 20 for 3* and 4* rated impact, ahead of Birmingham, Warwick and many of our competitors.