University of Wolverhampton: Toolkit for Access and Participation Evaluation (TAPE)

Over the past three years, HEAT has been committed to sharing the important resources other organisations are creating to support robust evaluation. In 2020, the Data team worked to integrate the Centre for Evaluation (CfE) Uni Connect Survey into the database, and in October 2022, TASO’s Widening Participation Questionnaire Scales were also shared with the membership through the Survey Tool.

Most recently, we reached out to Dr. Matthew Horton, Evaluation Manager at the University of Wolverhampton, to ask him to share more about his innovative survey, completed whilst undertaking his PhD at the University of Birmingham. This Toolkit for Access and Participation (TAPE) has been validated on over 1000 pupils, and it was found that pupils with higher scores (HE expectations, HE attitudes, HE knowledge and academic motivation) were significantly more likely to enter higher education (HE).

TAPE has now been peer reviewed within the Education Review Journal and can be accessed on the journal website. Dr. Horton has also provided an additional summary update of this work.

Evidencing ‘What Works’

The impact of this work speak volumes; TAPE is the largest study of its kind (to date), and the first to validate a toolkit for actual behaviours (such as HE entry). Furthermore, as an open source toolkit, TAPE has the potential to be particularly useful for smaller organisations who are limited by their capacity to dedicate resource to evaluation. TAPE is thus a compelling example of how sharing knowledge across the sector can foster collaboration and success.

We asked Dr. Horton some questions relating to the importance of TAPE as a collaborative tool, and how it can add value to HEAT and the wider sector.

What value do you see in TAPE as a collaborative tool?

“It makes sense to employ a validated toolkit to measure the success of your widening participation (WP) programmes and more widely across organisations to understand what does and does not work. TAPE and the validation work being commissioned by TASO will provide a validated bank of outcome questions. This will provide opportunities for the sector to compare the effectiveness of interventions in a standardised way.”

How do you advise member organisations utilise TAPE to maximise its potential?

“The TAPE survey has been developed to be employed as a pre-test-post-test design. This means that the survey should be completed before and after pupils engage in an individual intervention (e.g., mentoring) or programme of interventions. The survey provides evidence of whether pupil outcomes have improved as a result of the intervention (e.g., distance travelled).

I myself employed a quasi-experiment design (QED) to employ the toolkit. The survey was completed by the same pupils within schools before they engaged in WP activities and then a year or more later. Surveys were completed with whole year groups. This allowed outcomes to be tracked for those that had (treatment group) and had not engaged (a matched comparison group) in WP activities. This provided a straightforward way to collect consent for the comparison group, allowing wider outcomes to also be tracked (e.g., attainment and HE participation). This approach can provide type 2 or type 3 evidence, depending on how well-matched the comparison group is to the treatment group.”

What is the importance of such a large study for the sector?

“As outlined the TAPE toolkit has been validated via a wider quasi-experiment design that investigated the impact of a widening participation programme from 2012-2019. This is the  largest UK study to date and importantly;

1. Indicates that pupil HE expectations, HE knowledge, HE attitudes, and academic motivation are associated with whether or not they enter higher education. Despite WP programmes committing significant resources to improve such outcomes, there is a notable lack of robust evidence to support these claims of an association.
2. The validation work provides a robust toolkit that practitioners can confidently use to evaluate the effectiveness of their APP programmes and activities. This will strengthen our understanding of what does and does not work within programmes and across the sector.”

How do you envision TAPE adding value to HEAT?

“TAPE provides value by providing a robust validated online survey that can be employed by Higher Education Institutions (HEIs) to measure the impact of their work. This will allow HEIs to map their activity data to determine which interventions are most/least effective and if there are optimal points of engagement. A similar study has been completed by Burgess, Horton, and Moores (2021)[1] on HE entry and provides a useful guide on how analyse data WP data.”

TAPE Survey and guidance available now

Our Data team have built TAPE into the database using our Survey Tool. Please use the link below to access the Template:

TAPE (Horton, 2022) Survey Template (members only)

For additional information, please refer to Dr. Horton’s external guidance document published on File Store (members only).

We would like to extend our warmest thanks to Dr. Horton for kindly agreeing to share his findings and contact information with the wider membership. We hope to have more stories from our membership to spotlight and share in the future.

For more information on the Toolkit for Access and Participation Evaluation, please contact Matthew Horton at the University of Wolverhampton directly.