Website: Leeds Trinity University (https://www.leedstrinity.ac.uk)
No of Staff: 660 in academic year 2022/23 (according to HESA)
Their Challenge:
The aim of the project was to investigate the potential of Artificial Intelligence to support job evaluation in the Higher Education sector, and potentially more broadly. It was hoped that an AI tool could provide an unbiased consistent input that would reduce the time that job evaluation can sometimes take.
It was also hoped that this would give Leeds Trinity University students a real-life project to work on while undertaking their studies which would stand them in good stead for their future careers and support the University’s co-creation remit.
The Solution:
ECC used their expertise to train the project team on the HERA job evaluation scheme so that they would understand the rationale of job evaluation and ensure that no bias would be included in the technical build.  They also explained how the scoring methodology worked and helped the team to practice evaluating job roles. This training was a key component of the project before any technical build could commence.
The data features for machine learning were prepared and the project team focused on building a proof of concept using LTU’s data.
Fortnightly project governance/oversight meetings were scheduled to ensure progress was monitored. The project clearly showed the benefits of cross-organisational working, with HR and AI specialists working closely with MSc Data Science and AI and BSc Computer Science students.
Regular reviews of progress were held with ECC to ensure the integrity of the AI tool, and to preserve ECC’s intellectual property: although HERA was being used to demonstrate the proof of concept, the aim was to develop a tool that was not dependant on a specific job evaluation scheme.
During 2024 the system architecture of the proposed solution was developed and the workloads divided into:  
Work package 1:  Developing a Web server  
A HTML frontend was developed  
Python Flask based backend is under development   
Work package 2: Data collection, labelling  
Over 70 job descriptions were collected along with the corresponding job evaluation score against each element of the HERA scheme 
The language within the job descriptions were assigned to the relevant HERA element to enable the tool to recognise words and allocate appropriately 
A NoSQL database was designed (MongoDB)   
Work Package 3: Machine learning models 
A Natural language process API (Chatgpt API) had been selected to train AI models of the labelled data from Work package 2
A simple demo was developed to show the capabilities.
The Result:
The project has been a great example of co-creation with the team consisting of Professional, Academics and Students in coming together to develop the AI tool.
The tool so far has achieved a high level of consistency with the original job evaluation scoring undertaken by trained role analysts in the People and Culture team, showing that the training delivered by ECC to Academics and Students was understood and used to build the tool into understanding job descriptions and the weighting of the words.
The project has shown how AI can augment a service rather than replacing it, with AI proving to be a complementary tool to save time and release capacity. It was never the aim of the project to fully replace human interaction with job evaluation, and a human will always be needed to sign off the evaluation and check for consistency.
Next steps are to continue to train the AI bot to increase consistency before potentially leading to commercialisation of the product.
"We embarked on a project to explore the use of AI on job evaluation in universities funded by The Association of Commonwealth Universities. The project brought together HR, AI specialists working with our very talented students in our MSc Data Science and Artificial Intelligence programme and on our BSc Computer Science programme, and experts from ECC. This project offered our students opportunities to use the knowledge learned from their programme on this research project. We really valued the input into the project from ECC and their expertise to train the project team on the HERA job evaluation scheme so that they would understand the rationale of job evaluation and ensure that no bias would be included in the technical build.  ECC also explained how the scoring methodology worked and practiced evaluating job roles. This training was a key component of the project before any technical build could commence.
The project clearly showed the benefits of cross-organisational working.
ECC were involved throughout the project via regular reviews of progress to ensure the integrity of the AI tool and to seek their advice on how AI can support job evaluation.
At Leeds Trinity University we see research as an adventure, encouraging ideas and not being afraid to explore possibilities. We hope this pilot project inspires others to explore ways of realising the potential of their ideas through collaboration and innovative thinking and are grateful for the support we had from ECC on this journey of discovery."