Session 183 PD, Predictive Modeling for Actuaries: Machine Learning,

Predictive Modeling and Insurance Risk Management

Moderator:
Warren A. Manners, FSA, MAAA

Presenters:
Jeff T. Heaton
Anand Rao
Session 183 PD: Predictive
Modeling for Actuaries
Machine Learning, Predictive Modeling
and Insurance Risk Management
Using agent-based models
with Python and Repast
Overview of agent-based models (ABM),
with a short introduction to the Repast
framework and an explanation of how to
define agent actions with Python.
 What is an agent-based model?
Simulates complex systems of interacting
agents.
These agents might be customers,
companies, insurance agents or even
nations.
Agents must act inside of an environment.
Agents can deal with each other either
cooperatively, competitively or obliviously.

3
 How are ABMs Used?
Used to simulate large complex systems.
ABMs might be used to simulate and
model:
Interactions between policy holders and
competing agents.
Forecast effects of dynamic market conditions
on portfolios.
Butterfly effect interactions too hard to
predict without simulation.

4
 History of ABM
Cellular automata
1970 John Conway
introduces Game of Life
1996: Growing artificial
societies from the
ground up.
1997- Several implementations of
sugarscape from the book Growing
Artificial Societies from the Ground Up by
5
Epstein & Axtell.
 History of ABM (cont.)
ABM is used widely in
the video game
industry.
SimCity cities are
semi-autonomous.

1971, Schelling, T., Dynamic Models of

Segregation
2003, Karley, K., On the evolution of social
and organizational networks
6
 Simple ABM Example
Simple model of insurer response times to
meet varying consumer demand for 5
insurance products.
Two agent types: consumer and insurer.
Consumers demand 1 of 5 products. Once
demand is satisfied consumer will cycle to
the next product. (e.g. 1->2,,4->5, 5->1)
Insurers supply one product, and may
retool when half of requests are un-filled.
7
 ABM Example, cont.
Initial set of insurers offer random products
chosen uniformly.
Initial set of consumers demand random
products chosen uniformly.
Model will track the rise and fall of the
demand of each product on a linear plot.
Initial setup will be 10,000 consumer
agents and 10 insurer agents.
Experiment with different insurer counts.
8
 Create an ABM using Repast
Recursive Porous Agent Simulation Toolkit
(Repast)
Free open source (FOSS)
Supports Java, C#, Managed C++, Visual
Basic.Net, Managed Lisp, Managed
Prolog, and Python scripting.
Can be downloaded from:
http://repast.sourceforge.net/

9
 Repast, continued
Repast was originally developed by David
Sallach, Nick Collier, Tom Howe, Michael
North and others at the University of
Chicago.
Repast is widely used.
Google Scholar lists 50K citations and
references to Repast.
Used by the SOA for the research project
Simulating health behavior, by Alan Mills.
10
 Repast Architecture
Structure of
Repast ABM
Also typical
structure other
ABM systems.
Some systems
combine
Environment and
Model.
11
 What are Models & Environments?
Repast environment holds
system level information.
Repast models are what
some other systems call
the environment.
There is one instance of the model.
Fields hold data. Actions hold instructions.
Model fields & actions are global to all
agents.
12
 Example Model Design
The following field is defined at the model
level:
probNewProduct The probability that an
insurer will retool if it cannot fulfill 50% of its share
of the consumer demand.
consumersNumAgents The number of
consumer agents to use.
insuerersNumAgents The number of insurer
agents to use.
All three are configurable in that you can
modify them on the start model dialog,
without changing the code.
13
 Interpreting the Results
An agent model is very different than other
supervised models, such as linear
regression.
You are not training on input, with expected
outcome.
The outcome from an ABM is often a time
series prediction from the agents.
The outcome might be patterns in a time
series.
How you interpret these results is up to you.

14
 What is an Agent?
Agents are the central unit
of work in an ABM.
Models have one or more
agent types.
Many agent instances are
created for each agent type.
Fields define attributes unique to each agent
instance.
Actions define how agents update their
15
fields and interact with other agents.
 Consumer Agent Design
Consumer agents have the following
fields:
productDesired The product number 0-4,
that the consumer desires.
requestFilled Tracks how many requests
were filled.
requestFailed Tracks how many requests
failed.
One step action, to perform consumer
interactions with insurers.
16
 Insurer Agent Design
Insurer agents have the following fields:
currentProduct The current product the
insurer offers.
filledRequests The number of requests that
this insurer could fill this tick.
failedRequests The number of requests
that this insurer failed to fill this tick.
One step action, to perform insurer
interactions with consumers.
17
 Passage of Time

The simulation moves through ticks.

Actions and visualizers can fire on each tick,
intervals of ticks, or specific ticks.
Schedules defined at agent & model level
18
 Example Schedule
The schedule for the example model is
very simple, consisting of:
At every tick, update the visualization.
At every tick, allow consumers to demand
their products of the insurers.
At every tick, allow insurers to retool their
product offering, if desired.
There are no start and stop model actions.

19
 Example Actions
All actions were programmed in Repasts
Not Quite Python (NQPy) scripting
language.
The following actions are defined.
Model Setup
Consumer Step
Insurer Step
Visualization Step

20
 Model Setup Action
# Cause the customers to demand random products.
for consumer as ConsumerAgent in self.consumers:
product_num = Random.uniform.nextIntFromTo(0, 4)
consumer.setProductDesired(product_num)

# Cause the insurers to offer random products.

for insurer as InsurerAgent in self.insurers:
product_num = Random.uniform.nextIntFromTo(0, 4)
insurer.setCurrentProduct(product_num)

21
 Consumer Step Action
# Choose a random insurer to obtain the product from.
insurer_num = Random.uniform.nextIntFromTo(0,
self.model.insurers.size()-1)
insurer = (InsurerAgent)self.model.insurers.get(insurer_num)

# If the insurer has the product, then obtain it.

if insurer.getCurrentProduct() == self.productDesired:
insurer.setCash(insurer.getCash()+1)
self.requestFilled=self.productDesired
self.productDesired = self.productDesired + 1
insurer.setFilledRequests(insurer.getFilledRequests()+1)
# Change our product desired, simply cycle between 0 and 4.
if self.productDesired>=5:
self.productDesired=0
else:
# If the insurer does not have the product, then record that.
insurer.setFailedRequests(insurer.getFailedRequests()+1)
self.requestFilled=-1
22
 Insurer Step Action
# Did we fail to fulfill any orders (prevent div/0)?
if self.failedRequests>0:
ratio = self.filledRequests / self.failedRequests

# Did we fail to fulfill 50% of the requests?

if ratio < 0.5:
refit = Random.uniform.nextDouble()

# Do we want to retool?
if refit<self.model.probNewProduct:
self.currentProduct =
Random.uniform.nextIntFromTo(0, 4)
self.cash = self.cash - 5

23
 Visualizer with 10 Insurers

This visualizer shows a model with 10,000

consumer agents & 10 insurers.
Product demand rises and falls as insurers
24
adjust to the changing demand.
 Visualizer with 2 Insurers

The above model had 2 insurers and 10,000

consumer agents.
With 2 providers, consumer demand is
25
forced into smaller tiers & sharp transition.
 Code Examples
The Repast model from this presentation
can be found at the following URL.
https://github.com/jeffheaton/soa

26
 Contact Details

JeffHeaton
DataScientist,RGA

Email:jheaton@rgare.com

Twitter:@jeffheaton

27
Behavioral simulation of
policyholder behavior
Overview of how agent-based models
(ABM) can be combined with behavioral
economics to better understand investment
decisions made by consumers
 What is behavioral simulation?
ArtificialIntelligence AgentBasedModeling
Cognitivethoughtthrough
machines BehavioralSimulation

ComplexSystems
Emergentsystem
behaviorfromindividual
actions
Sophisticated,computationally
ComputationalPower intensivemodelingtechniquethat
Rapidcycletime reliesuponadecentralizedsetof
forintensivecalculations behavioralrulesandstudies
emergentbehaviors

ClassicalEconomics BehavioralEconomics
Individualdecisionmaking
drivenbyselfinterestand Simulationofhowindividualsreally
utilitymaximization makedecisionsandtheiremergent
groupbehaviorsbasedonmodeling
Psychology individualbehaviorsasagents.
Scientificstudyofmental Studyofindividualdecisionmaking
functionsandbehaviorsof basedoncognitive,heuristic,
individualsandgroups emotionalandsocialfactors

29
 SimCity for Financial Services

30
 What is modelled?
Households FinancialAdvisors

Economy Health

PlanSponsors FSIs

31
 Data Fusion
Narrow & Broad & Synthetic
Deep Datasets Shallow Data Population

+ =

Surveys e.g., SBIs Market Data e.g., US HHBS/IE Data for Synthetic US
MacroMonitor Data Census Population
4,000-5,000 households 320 Million households Millions of households
100s of variables 10s of variables 100s or 1000s of variables

Deterministic Stochastic
Non-Parametric Parametric

Nearest neighbor algorithm Conditional mean matching

Hot-deck Imputation Markov Chain Monte Carlo
Bayesian Data Augmentation

32
 Data Sources
SupportingHHBSDataSources:
1. BureauofLaborStatistics(BLS) ConsumerExpenditure
Survey(CES)ofUShouseholdsannualexpendituresforfood,
clothing,shelter,health,utilities,transportation,supplies,
entertainment,etc.
2. StrategicBusinessInsights(SBI)MacroMonitor
ComprehensiveprojectablesurveyofUShouseholdsfinancial
needs,demographics,products,services,channels,and
attitudeswithnearly4,000variables.
3. EmployeeBenefitsResearchInstitute(EBRI)
4. NationalBureauofEconomicResearch(NBER)
5. RetirementIncomeIndustryAssociation
6. Othersources SocialSecurity,andothersourcesofsocio
economic,healthcare,andretirementdata.

33
 Data and Behaviors
IncomeStatement Households BalanceSheet

Income Assets
Salary Home
BusinessOwnership Car
Annuities SavingsAccount
Dividends IRA
CapitalGains 401(K)
SocialSecurity Annuity
Inheritance Pension
Etc. Etc.
Behaviors
Consumption Liabilities
Discretionary clothes, Buyingahome Mortgage
restaurants,vacations, Findingajob Carloans
privateschool,etc. Spending Consumerloans
Nondiscretionary Saving Creditcarddebts
mortgage,basic Investing Otherrealestateloans
groceries,healthcare, Planning Educationdebts
insurance,etc. Retiring Healthcaredebts

34
Dynamic Cradle-to-Grave Simulations
Thesystemmodels:

IndividualLifeCycle
Interactionofindividualswith AssetCycle
theiradvisors;

Stages
Changesingovernmentsocial
programssuchasSocial Dependent
Single& Growing
PreRetiree Retiree
New
Rich Family Generation
SecurityandMedicare;and
Changesinthetaxsystem. LiabilityCreation AssetDepletion

Advice AssetCreation AssetCreation

Othernaturalextensionsare
AssetProtection AssetTransfer
modeling
Forexample,modelingthe AssetPreservation

behaviorsofretiredindividuals PrimaryFocus OtherFactors Scenarios

EnvironmentalFactors

ifthecurrentlowinterestrate 1. LifeCycle 1. Advisors 1. ProlongLowinterest

2. Economy 2. SocialPrograms RateEnvironment
environmentcontinuesfora 3. TaxSystem 2. HighInflationary
prolongedperiodoftime Environment

35
 Life-cycle Stages
WehavedividedthelifecycleofAn
individualintosixstages:

IndividualLifeCycle
AssetCycle
1. Dependents

Stages
2. Single&Rich
3. GrowingFamily Single& Growing New
Dependent PreRetiree Retiree
Rich Family Generation
4. Preretiree
5. Retiree
LiabilityCreation AssetDepletion
6. NewGeneration
Advice AssetCreation AssetCreation
Ourgoalistomodelhowthelifeevents
andthechoicesoftheindividualschange AssetProtection AssetTransfer
overtimetoreflectvariouslifestages
suchaswhen: AssetPreservation

1. Heorshegraduatesfromcollegeand Payingoffstudentloans Payingtuitionbills

EnvironmentalFactors

findsajob Startingacareer Caringforparents

Planningforretirement
2. Heorshemarries,buysahome,and
havechildren Gettingmarried Withdrawalmoneyfor
Buyingahome retirement
3. Theirchildrengotocollege,andthey
Havingoradoptingchildren Payingforhealthcare
begintakingcareoftheirparent(s)
Creatingalegacy
4. Theybecomeemptynesters,and
thentheyretire

36
 Consumer Behavior Model
ACTIONABLEINSIGHTS

MarketLevelInsights HouseholdSimulations ProductLevelInsights

ANALYSIS&SYNTHESIS

Synthetic Behavioral Once upon a time Once upon a

time Once upon

Population Simulation

Household Whatif?
ScenarioBuilding
Fundedness

DATAINPUTS
HHDemographic LifeEvents HealthcareCost

HHFinancials MacroEconomic ProductFeatures

37
 Modeling Home Purchase Decisions
Familysize Savingsversusdownpayment
Rentversusmortgage Desiredhousesize
Startafamily
1 Anindividualbeginsthinkingabout
Stableemploymentprospect Taxeffects
purchasingahome,withmanyfactors
influencingthedecision.

2 Whentheconditionsareright,theperson
willpurchaseahome.Hewilldrawfromhis 1 Rent
savingstopayforthedownpaymentand
fees.Hisrentalexpensesgoingforwardare
replacedbymortgagepayments.
2 Ownhome
3 Ashiswealthincreasesandchildrenare
born,hemaydecidetobuyalargerhome.
3
4 Ifacashneedarises(e.g.medicalexpenses),
hemaydecidetosellthehouseandreturn
torenting. Upsize Downsize

4 Sell Reversemortgage

38
 Modeling Employment Decisions

1 Consideranindividualwhojustretired,buthisincome
barelycovershisexpenses.
5

2 Ifhisexpensesincrease(e.g.,becausehiswifehas
unexpectedhealthcarecosts),thenhemaybeforced Employed
tolookforajob.

3 Hewillbeconsideredunemployedwhilehesearches
forajobtocoverhisextraexpenses.
4 6
4 Hewillthenfindanewjobwithaprobabilitybasedon
hisage,occupationandthecurrentstateofthe
economy. Unemployed Retired

2
5 Whileheisemployed,hewillhaveenoughincometo 3 1
coverthesenewhealthcarecosts.

6 Onceherillnesspassesandthehealthcarecostsdrop
orhebecomestooilltowork,hewillreturnto
retirement.

39
Simulating Individuals and Households

40
 Life-event impacts

HouseholdEvents

IndividualEvents

41
Macro-view of household structures

42
 Where can we use Behavioral
Simulations?
Products, Capital,
Strategy& Customer& Sales& Process& Inforce
Pricing& Risk&
Growth Marketing Distribution Operations Management*
Underwriting Finance

Howwillchangingconsumer
demographicandsocio
economicforcesimpactdemand
forourproducts?
Howcanweimprovepolicyholder
persistency?
Doourproductdesignsreflect
theevolvingneedsofour
customerbase?

* Including Claims & Benefits

43
 Example #1: Withdrawal
Considerwhenthereisacashneed,andthe
individualmustmakeadecisionofwhetherornot
tomakeawithdrawalfromtheirvariableannuity Savings
contract.Thisdecisionwilldependon:
1. Theirfamilysituation;
CDs
2. Theirfinancialliteracy;
3. Howtheproductwasframedwhenitwassold Whichasset
tothem; doImakea
withdrawal MutualFunds
4. Whatothertypeoffinancialassetstheyown; from?
5. Whatarethetaxconsequences;and
VariableAnnuity
6. Howmuchthevariableannuitycontractisin
themoney. Individual
Withregardstothelastcriteria,consideration 401(k)
mayalsobegiventovariouscontractprovisions
suchasthesurrenderpenaltyandwhetherthe
guaranteedminimumbenefitofthevariable
annuitycontractisstillinthewaitingperiod.

44
 Withdrawal decision process
Cashneedfulfilled
6

Cashneed Otheraccounts
2 unfulfilled (CD,mutual
Event
funds,401k)
(i.e.,healthissue)
Needcash Account
1 4 withdrawal
Policyholder Consideration hierarchy PartialVA
dormant 3 ofwithdrawal withdrawal
Usedisposable 5
income
Cashneed FullVA
covered withdrawal

Whileheisretiredandhisfixedincomecovershis Ifhedoesnotbelievehissourcesofincomewillcoverhis
1 expenses,hewillremaindormantwithnofinancial 4
expenseduringthetimeheisjobsearching,hewillbeginto
concerns. worryandconsiderwithdrawingcashfromhisinvestments.

Whenhiswifegetssick,hewillcalculatehowmuch Ifhedecidestowithdraw,hewillfollowawithdrawal
2 5
moneyhewillneedtocoverhermedicalbills. hierarchy,tappingintooneaccountatatimeuntilhehas
fulfilledhiscashneed.
Whileheislookingforajobtocoverhermedicalbills,he Oncehiscashneedisfulfilled,hewillreturntothedormant
3 6
willcalculatehowlongtheycanliveoffoftheircurrent state.
incomesources.

45
 Annuity Lapses & Withdrawals

46
 Examples #2: Retirement Readiness
MacroEnvironment Decisionmaking

Economy Householdstructure
Inflation Assetallocation
Unemployment Indebtedness
Returnonfinancial Health
assets SocialSecurity
Homevalues Retirementage
Health
Lifeexpectancy
Outofpocket Fundedness
healthcarecosts
Highereducationcost Fundedness: RatioofExpectedRetirementSavingsVs.
anddebt ExpectedRetirementExpenses(Overfunded:>110%;
SocialSecurityoutlays Constrained:70110%;Underfunded:<70%)
GlidePath:Portfoliomixchangesthroughlifestagesand
economicconditions

47
 Uses of Retirement Readiness
FitnessMaps OpportunityMaps GlidePathAnalysis

Marketlevel Marketlevel Marketlevel

Howpreciselycanwe Howdoestheaveragenet Whatwillbetheimpactofa
estimatethepreparednessof worthoftheAffluentBuilder cashheavyportfolio
MiddleMarketBuildersfor segmentchangeunder allocationonthefundedness
retirement? differenteconomic ofAffluentsegments?
scenarios?

Companylevel Companylevel Companylevel

Towhatextentwillthe Whatisthepotential Whatfundmixshould
fundednessoftheemployees adoptionrateand recommendtomyAffluent
inmyplanimproveifIadd investmentrateofmytarget customers,whileensuringa
theoptiontobuyannuities? datefundamongmyAffluent monthlyreservecushionof
Buildercustomers? atleast$5,000throughthe
ageof90?

48
 Scenario Analysis
BaseScenarioResults Percentagesaddupto100%acrosseachsegment
Household(%) MedianNetWorth($K)

LifeStage UF C OF UF C OF
Starters 83% 15% 2% $21 $868 $5,581
Builders 80% 7% 13% $195 $1,086 $4,123
PreRetired 66% 13% 21% $150 $601 $2,691
Retired 64% 9% 27% $213 $1,422 $2,429

Wealth UF C OF UF C OF
Marginal 90% 9% 1% $38 $333 $915
MassMarket 65% 16% 19% $392 $957 $1,335
Affluent 25% 10% 65% $2,733 $2,416 $5,356
Wealthy 3% 5% 92% $6,131 $5,459 $11,118

Acrossalllifestages,64%to83%areUnderfunded
evenamongtheRetired,amajorityofthepopulationisunderfunded.
MedianNetWorthofHHsinsamewealthsegmentishigherforoverfunded...
anddifferenceofnetworth forunderfundedandoverfundedrangesfrom900Kto5Mn.

49
 Retirement Heatmap

50
 Conclusion
Dynamicsimulationofconsumerfinancialbehaviorof
representativeUSpopulationfromcradletograve
Exploitsmacrolevelandmicrolevelmodeling
Granularindividualandhouseholdlevelbigdataasopposed
toaggregates
Builtusingcommercialstrengthsoftware(i.e.,AnyLogic)
capableofrunning500,000+consumeragents
Platformthatallowsanumberofbehavioralandmacro
economicpoliciestobeanalyzedandevaluated

51
 Contact Details

Dr.AnandS.Rao
Partner,PwCAdvisoryServices
InnovationLead,AnalyticsGroup

Email:anand.s.rao@us.pwc.com

Twitter:AnandSRao

52