The data streams are focused towards a user base that would typically read a single advert at a time. This means that categorisation of the data in to Office of National Statistics Standard Occupation Codes is required. Given that the numbers of records exceeds 1 million and is growing daily, this is not possible manually and an automated solution is required.

We developed an automated classifier algorithm using a random forest decision tree Machine Learning approach. Based on our back testing, the classifier is operating at greater than 99% accuracy of classification of a job based on the job advert text alone. In addition to this classifier, we also developed the capability to extract additional information about each job, such as employer, location, time of advertisement and, if available, salary being offered. Repeated application allows us to generate time series and comparative analysis of jobs in a variety of area types (LEP, Local Authority as well as custom areas) across the whole of the UK.