Lifestyle Article - How to read 18,701 documents in less than 2 hours


We read 18,701 job adverts to answer the question: what does an actuary do? Reading this many adverts should have taken weeks, so how were we able to do it in 2 hours? Machine Learning! We read the data into R and let a machine learning algorithm called “Topic Modelling” do the hard work. What we found was very surprising….

 The algorithm identified 40 different “topics”. We can explore the meaning of each topic using an interactive visualisation – to start interacting click here.
 

 Clicking on bubble 32 allows us to reveal the most relevant words for that topic which are displayed below. This topic is best described as “Investment & ALM”.

 

 Have a play with the visualisation and see how many topics you can identify.

 What does an actuary do?
 The algorithm automatically identified the major types of Actuarial work including Speciality Pricing (5) Capital Modelling (6/29) Pensions (10) Annuity Pricing (11), Reserving (13), Reporting (15) Personal Lines Pricing (16) Risk Management (18) Life (19/24) Prophet Modelling (21) Solvency ii (25) Catastrophe Modelling (27) Investment and ALM (32) and Marine/Aviation (35). The algorithm also detected softer skills such as Project Management (1) Leadership (2) as well as experience level such as Junior(36), Making Progress through exams (3), Newly/Nearly Qualified(20) and Seniority (7).

 Expect the Unexpected
 Having delivered several unstructured data projects in different industries one learning point stands out – expect the unexpected! Unexpected results include
 • Online Casino Chat Logs – “We didn’t realise customers were having difficulty depositing with their credit card!”
 • Insurance Company Loss Descriptions – “We shouldn’t be paying claims for that event– it’s not covered by the policy!”

 Still we were surprised when we clicked on topic 26 to see a topic dedicated to French actuarial roles! Furthermore when we investigated the topics in the top right of the bubble chart we were fascinated to see they related to the various actuarial recruitment companies! The algorithm had managed to automatically differentiate between the role requirements and the recruiters “sales pitch”.
 
 What did we learn?
 Since the role of an actuary is well understood, we have essentially validated our machine learning approach as a method of discovering new and reliable patterns from new, unexplored data items. The next time we want to quickly understand a large collection of text documents we should consider using this approach. To learn more about this fascinating subject click here

 Machine Learning Solutions provide training to actuaries in Machine Learning techniques as well as delivery of technical projects in predictive analytics across various industries. For more information visit here 
  

Back to Index


Similar News to this Story

LCP appoints Holly McArthur as Head of Sole Trusteeship
LCP has appointed Partner Holly McArthur as Head of Sole Trusteeship in response to the steep growth in this area of the professional trustee market.
PIC appoint two new independent Non Exec Directors
Pension Insurance Corporation plc (“PIC”) have announced that Martin Pike has joined PIC’s Board as an Independent Non-Executive Director, and that Tr
Aon announce the appointment of John Neal
New global climate role will help tackle increasing weather and catastrophe risks through advanced analytics, innovative solutions and new sources of

Site Search

Exact   Any  

Latest Actuarial Jobs

Actuarial Login

Email
Password
 Jobseeker    Client
Reminder Logon

APA Sponsors

Actuarial Jobs & News Feeds

Jobs RSS News RSS

WikiActuary

Be the first to contribute to our definitive actuarial reference forum. Built by actuaries for actuaries.