Models for MI Group

Author

Philip L. Adams

1 Purpose for Analysis

The Society of Actuaries is interested in better understanding whether life insurance data, as currently provided in the Individual Life Experience Committee’s (ILEC) data compilation, can be used to calibrate trends in industry mortality. Life actuaries have avoided using this dataset for trend analysis. Significant heterogeneity underlies the ILEC data which is known to obscure the true underlying mortality trend. Factors known to alter the composition of the dataset over time include changes in underwriting practices across face amount and product type, changes in company mix, and changes in target markets of individual companies. Moreover, the latest public dataset covers 2009-2017, a window that is too short for credible and reliable estimates of long-term trend. Therefore, life insurance actuaries have relied on population-based mortality trend models despite the basis risk.

Mortality analysis in life insurance is almost always done against an industry-standard table whenever appropriate. Actuaries tend to rely on the most recent tables such as the 2015 Valuation Basis Table (2015VBT). This table provides mortality rates by issue age, sex, smoker status, and policy duration and was calibrated using data collected for observation years 2002-2009. Other known drivers of mortality, like face amount and underwriting, are not included. Therefore, assessing secular mortality trends against this table is challenging, since any observable trend will be the combination of underlying secular trends and of shifts in the prevalence of drivers not included in the table. For example, if the distribution of face amounts shifts higher much faster than inflation, then average mortality is expected to decline over time against the 2015 VBT given that mortality declines with increasing face amount in well-underwritten portfolios.

The aim of what follows is to provide a mortality model which “raises the floor” of analysis away from the 2015VBT. The model aims to control for additional drivers not yet accounted for in industry tables and adjust for changes in known drivers. The new benchmark is then available for an actuary to attempt additional analysis of mortality trend in the ILEC dataset.

Conspicuously absent is any provision for mortality trend in this analysis. This is intentional. The paper which this model supports is intended to explore and compare the observed trends in the ILEC data against the same information in population trend studies. Additionally, calibrating a model for trend is subject to significant research on its own:

  • The traditional approach is to use variants of Lee-Carter models. The current regression approach can be extended using mixed-effects modeling or Bayesian methods.
  • One can also use splines of varying complexity. For example, in prior corporate work, I have used the mgcv package to model time trends which vary by attained age where the spline is based on attained age and the “by” parameter of the spline is set to calendar year. This has the effect of setting a custom slope for the trend line by attained age.

Other authors have taken a variety of other approaches, and search engines and LLMs are available to learn more.

Given the many potential shapes, I opted to leave the analysis of trend to the main body of the study. Nonetheless, a high-level illustration of the impact on trend of controlling for other drivers is provided toward the end of this document.

2 Summary of Results

Two different types of models were built. One type based on boosted decision trees was used primarily for exploratory data analysis. The other type based on elastic net regression was used to adjust the 2015 VBT table.

Selected findings from the boosted decision trees:

  • In general, there are substantial differences between low face term and high face term prior to the end of the level term period, defined as below or above 100,000 of face amount, and interactions among variables are most prevalent in the low face amount term.
  • Significant elevations are noted in 2-class preferred systems in the term subset.
  • Curiously, the subset of term other than post-level term exhibits multimodality in the distribution of SHAP values in certain cases, such as with residual standard of the 2-class preferred systems. This suggests heterogeneity in the subset.

Selected findings from the elastic net modeling:

  • Five models were necessary, one each for post-level term, low face other than post level term, high face other than post level term, permanent unismoke, and other permanent business.
  • The model for low face other than post level term was the most complicated with the greatest number of interactions.

At the end of this document is a discussion of the impact on mortality trend of adjusting out unmodeled variation. The per annum trend of ratios against the 2015 VBT is approximately -1.5%, while the same against the elastic net model is -0.8%. While not the sole reason for the movement, we note the rapid upward shift in UL/VL exposures over the same time period. These patterns provide evidence that the nominal shift is in part due to shifts in face amount distribution. Other shifts in prevalence can be found, and mortality trends vary by subset both before and after adjusting for unmodeled variation.

3 Setup

The following sets up the environment for further analysis.

Code
library(catboost)
library(data.table)
library(tidyverse)
library(dtplyr)
library(shapviz)
library(patchwork)
library(doParallel)
library(glmnet)
library(openxlsx)
library(flextable)
library(ftExtra)
library(Matrix)
library(MatrixModels)
library(ggridges)
library(arrow)

source('catboost.shapviz.R') # Function which adapts catboost to shapviz
source('glmnet_support.R') # Support functions
source('support_fns.R')

bUseCache <- TRUE # Use cached objects where available
bInvalidateCaches <- FALSE # Force computation even if cached

training.fraction <- 0.7 # Fraction of data for boosting

cb.task.type <- "GPU" # Use the GPU, set to "CPU" if needed

nFolds <- 10
cvfit.seed <- 13579
catboost.seed <- 12321
traintest.seed <- 1337

fit_params <- list(iterations = 50000,
                   task_type=cb.task.type,
                   loss_function = 'Poisson',
                   verbose=250,
                   od_pval=0.05,
                   random_seed = catboost.seed)

# Labels for relabeling face amount for nicer looking plots and tables
fa.remap <- data.table(
  Face_Amount_Band.Old = c(
    "01: 0 - 9,999",
    "02: 10,000 - 24,999",
    "03: 25,000 - 49,999",
    "04: 50,000 - 99,999",
    "05: 100,000 - 249,999",
    "06: 250,000 - 499,999",
    "07: 500,000 - 999,999",
    "08: 1,000,000 - 2,499,999",
    "09: 2,500,000 - 4,999,999",
    "10: 5,000,000 - 9,999,999",
    "11: 10,000,000+"
  ),
  Face_Amount_Band.New = c(
    "<10K",
    "10K - <25K",
    "25K - <50K",
    "50K - <100K",
    "100K - <250K",
    "250K - <500K",
    "500K - <1M",
    "1M - <2.5M",
    "2.5M - <5M",
    "5M - <10M",
    "10M+"
  )
)

4 Data Prep

The data were extracted from the ILEC dataset and restricted to experience years 2011-2017. The data prep steps are included in the embedded code but not executed in this document.

4.1 Term Data Preparation

Code
library(arrow)
library(data.table)
library(tidyverse)
library(dtplyr)

ilec_arrow_src <- '/workspace/Projects/ILEC/VBT/Data/ilecdata'

dat.arrow <- arrow::open_dataset(ilec_arrow_src)

# Term No PLT

dat.arrow %>%
  filter(Observation_Year >= 2011 
         & Observation_Year <= 2017 
         & Insurance_Plan == 'Term'
         & SOA_Post_Lvl_Ind != 'PLT') %>%
  select(!Observation_Year) %>%
  group_by(
    Sex,
    Smoker_Status,
    Attained_Age,
    Duration,
    Face_Amount_Band,
    SOA_Post_Lvl_Ind,
    Number_of_Pfd_Classes,
    Preferred_Class
  ) %>%
  summarize(
    Death_Count=sum(Death_Count),
    Policies_Exposed=sum(Policies_Exposed),
    ExpDth_Cnt_VBT2015=sum(ExpDth_Cnt_VBT2015),
    Death_Claim_Amount=sum(Death_Claim_Amount),
    Amount_Exposed=sum(Amount_Exposed),
    ExpDth_Amt_VBT2015=sum(ExpDth_Amt_VBT2015)
    ) %>%
  collect() %>%
  as.data.table() -> 
  dat.term.noplt

dat.term.noplt[,
         `:=`(Smoker_Status=as.character(Smoker_Status),
              Number_of_Pfd_Classes=as.character(Number_of_Pfd_Classes),
              Preferred_Class=as.character(Preferred_Class))]

dat.term.noplt[is.na(Number_of_Pfd_Classes),
         Number_of_Pfd_Classes:="U"]

dat.term.noplt[is.na(Preferred_Class),
         Preferred_Class:="U"]

dat.term.noplt[,UW:=paste0(Smoker_Status,"/",Preferred_Class,"/",
                           Number_of_Pfd_Classes)]

saveRDS(dat.term.noplt,"dat.term.noplt.rds")

# Term With PLT

dat.arrow %>%
  filter(Observation_Year >= 2011 
         & Observation_Year <= 2017 
         & Insurance_Plan == 'Term'
         & SOA_Post_Lvl_Ind == 'PLT') %>%
  select(!Observation_Year) %>%
  group_by(
    Sex,
    Smoker_Status,
    Attained_Age,
    Duration,
    Face_Amount_Band,
    SOA_Post_Lvl_Ind,
    Number_of_Pfd_Classes,
    Preferred_Class,
    SOA_Antp_Lvl_TP
  ) %>%
  summarize(
    Death_Count=sum(Death_Count),
    Policies_Exposed=sum(Policies_Exposed),
    ExpDth_Cnt_VBT2015=sum(ExpDth_Cnt_VBT2015),
    Death_Claim_Amount=sum(Death_Claim_Amount),
    Amount_Exposed=sum(Amount_Exposed),
    ExpDth_Amt_VBT2015=sum(ExpDth_Amt_VBT2015)
  ) %>%
  collect() %>%
  as.data.table() -> 
  dat.term.plt

dat.term.plt[,
               `:=`(Smoker_Status=as.character(Smoker_Status),
                    Number_of_Pfd_Classes=as.character(Number_of_Pfd_Classes),
                    Preferred_Class=as.character(Preferred_Class))]

dat.term.plt[is.na(Number_of_Pfd_Classes),
               Number_of_Pfd_Classes:="U"]

dat.term.plt[is.na(Preferred_Class),
               Preferred_Class:="U"]

dat.term.plt[,UW:=paste0(Smoker_Status,"/",Preferred_Class,"/",
                         Number_of_Pfd_Classes)]

saveRDS(dat.term.plt,"dat.term.plt.rds")

4.2 Perm Data Preparation

Code
library(arrow)
library(data.table)
library(tidyverse)
library(dtplyr)

ilec_arrow_src <- '/workspace/Projects/ILEC/VBT/Data/ilecdata'

dat.arrow <- arrow::open_dataset(ilec_arrow_src)

# Term No PLT

dat.arrow %>%
  filter(Observation_Year >= 2011 
         & Observation_Year <= 2017 
         & Insurance_Plan != 'Term') %>%
  select(!Observation_Year) %>%
  group_by(
    Sex,
    Smoker_Status,
    Attained_Age,
    Duration,
    Face_Amount_Band,
    Insurance_Plan,
    Number_of_Pfd_Classes,
    Preferred_Class
  ) %>%
  summarize(
    Death_Count=sum(Death_Count),
    Policies_Exposed=sum(Policies_Exposed),
    ExpDth_Cnt_VBT2015=sum(ExpDth_Cnt_VBT2015),
    Death_Claim_Amount=sum(Death_Claim_Amount),
    Amount_Exposed=sum(Amount_Exposed),
    ExpDth_Amt_VBT2015=sum(ExpDth_Amt_VBT2015)
    ) %>%
  collect() %>%
  as.data.table() -> 
  dat.perm

dat.perm[,
         `:=`(Smoker_Status=as.character(Smoker_Status),
              Number_of_Pfd_Classes=as.character(Number_of_Pfd_Classes),
              Preferred_Class=as.character(Preferred_Class))]

dat.perm[is.na(Number_of_Pfd_Classes),
         Number_of_Pfd_Classes:="U"]

dat.perm[is.na(Preferred_Class),
         Preferred_Class:="U"]

dat.perm[,UW:=paste0(Smoker_Status,"/",Preferred_Class,"/",Number_of_Pfd_Classes)]

saveRDS(dat.perm,"dat.perm.rds")

4.3 Data Adjustments

Variables like attained age and duration were discretized into relatively small buckets. Because of the aggregation bias in boosted decision trees when analyzing feature importance, any variable with a lot of levels will tend to “hoard” feature importance, especially if they have a lot of variability between levels of the predictor.

  1. Attained age was broken into quinquennial ranges, with 0-17 and 18-25 being the exceptions.

  2. Duration was broken into buckets whose width increased with duration. Thus, there are singleton groups for durations 1, 2, and 3, a doublet for 4 and 5, then quinquennial through duration 20, decennial to 30, and one large bucket thereafter.

There were other minor changes.

  1. Face amount band has had labels adjusted for better plotting and rendering in tables.

  2. The smoker status, number of preferred classes, and preferred class was combined into a single factor where applicable.

  3. The data were assigned into training/test partitions.

  4. A/E ratios were computed.

  5. A Noise column of standard normal random samples was added. While this was a by-product of some bug testing with catboost, it proved to be an interesting addition as a predictor for boosted decision trees. It provides a useful frame of reference when comparing variables in the feature importance plots.

It became apparent during modeling that the data should be subdivided. There are three reasons for this.

  1. There are differences in the underlying variable structure. Term length and pre-/post-level term information are populated for Term insurance plans but meaningless for the other insurance plans. Duration also has a different semantic interpretation in the presence of term length.

  2. There are imbalances in the exposures or claims. The subset of unismoke Perm experience has total claims which dwarf the rest of the non-term insurance plans. For perm, there is also far less 4-class non-smoker coverage than there is for the 3-class, while term carries a meaningful amount of 4-class relative to 3-class non-smoker.

  3. A sufficiently different mortality pattern exists, such that a separate model is more efficient than a unified model. This occurred in the no-post-level-term Term model, where a single model had difficulty fitting for face amounts under 100,000.

Therefore, there are five models in four subdocuments.

  1. Term other than PLT, PLT term, unismoke Perm, and the smoker-distinct permanent plans

  2. Within term other than PLT, there are separate models for under and over 100K face amounts.

5 How To Read This Document

This is laid out similarly to a traditional modeling exercise where the data are prepared, the models are built and evaluated from a statistical perspective, and results and other outputs are available at the end of the process. There is no need to read it linearly. It might useful to start at the results (Factor Tables, Plots of Terms, Tables of Terms, Goodness-of-Fit tables) before approaching other parts. The table of contents to the left expands down to those sections.

Each subdivision is formatted the same way, although some details may differ:

  1. Feature Discovery, where gradient boosting is applied to the data.

  2. Feature Importance, where the univariate and bivariate important features are explored.

  3. Elastic net modeling, where data are prepped and models calibrated. These include typical plots for the cross-validation and the coefficient shrinkage by penalization parameter.

  4. Factor Table, where a table of non-trivial factors from the model are presented. These are slates of credible factors.

  5. Plots of Terms, where the predicted terms are plotted together and variables not part of the plot are fixed at certain values.

  6. Tables of Terms, same as the plots, but in tabular form.

  7. Goodness-of-Fit Tables, where actual-to-model ratios are shown for univariate and bivariate slices of the data.

6 Brief Introduction to the Model Types

6.1 Choice of Models

While survival modeling, specifically Cox modeling, is the norm in other disciplines, Poisson regression is frequently used in mortality modeling due to the structure of actuarial experience studies. A justification for why Poisson regression models are equivalent to Cox models and therefore acceptable for mortality modeling can be found in Whitehead (1980). More complex models were not considered given the need for transparency and explainability.

6.2 Gradient Boosting

Gradient boosting is a technique which recursively fits models of the same type on a dataset until some stopping rule is reached. Here, gradient boosted decision trees are used. First, a shallow decision tree is fit to the data using a random subset of columns, where the tree is relatively shallow. The fitted values are applied and removed from the response variable, and another shallow decision tree is fit on those residuals. On and on this goes, until it hits a stopping rule. In this case, the stopping rule is when the prediction error on an out-of-sample subset increases for more than a specified number of iterations.

6.3 Interpretation Example for Gradient Boosting

Gradient boosting can produce SHAP values for each observation or row of covariates. For our purposes, the SHAP value for a given row of data and given covariate can be thought of as the marginal contribution of that covariate to the outcome for that row of data. There is additionally a baseline term analogous to an intercept in GLMs.

Consider the first row of SHAP values for the Term, no PLT model. The exponentiated baseline of the model is 81.9%, compared to the A/E vs 2015VBT of 83%. For this row, face amount band and underwriting increase the predicted factor above 100%.

Variable Name Variable Value SHAP Value Exponentiated SHAP
Noise 0.189766 0.00275 100.3%
Sex F -0.02366 97.7%
Face_Amount_Band 03 - 25,000-49,999 0.166174 118.1%
SOA_Post_Lvl_Ind ULT 0.004194 100.4%
UW NS/U/U 0.091807 109.6%
AA_Grp 61-65 -0.00502 99.5%
Dur_Grp 6-10 -0.02374 97.7%
Baseline -0.19927 81.9%
Total Prediction 0.013243 101.3%

In isolation, these can be interesting in an underwriting context. However, to understand the broader picture, we use summary statistics.

Variable importance plot displays the average of absolute values of SHAP values for a given variable. This differs from feature importance which measures how much variability in the outcome is attributable to that factor. It is neither a measure of statistical significance nor an analysis of variance through. If a variable participates in lots of splits, then it will get partial credit for the variability explained by other variables in the trees.

The addition of the Noise term to feature importance creates a baseline of comparison. Noise is uncorrelated Gaussian noise. Thus, if another feature has similar SHAP patterns as to noise, there may be reason to believe it is not as influential or significant. One should be cautious about using this to dismiss a variable. For example, Sex appears similar to Noise in some cases in the one-dimensional plot. Yet, it appears with another variable in a highly influential interaction.

Importance of interactions is also extracted and shown. The table of scores is not informative. However, more informative are the plots of the distributions of SHAP values. If present, these can reveal shapes in the data, and if the shapes differ from subgroup to subgroup, then that is strong visual evidence for an interaction. Interesting potential interactions are pointed out in the description of the plots.

6.4 Elastic Net GLMs

Elastic net regularization allows the modeler to combine both LASSO and ridge penalties into a single model.

Ordinary least squares regression requires minimizing the squared difference of the response variable and the predicted values. In symbols,

\[ \underset{\beta}{\arg\min} \sum_{n}(y-X\beta)^{2} \]

This is equivalent to maximum likelihood estimation, where one assumes that the response variable \(y\) is normally distributed with mean \(X\beta\) and variance \(\sigma^{2}I_{k \times k}\). The maximum is taken with respect to \(\beta\), and the variance parameter is assumed to be fixed but unknown.

The LASSO and ridge regression methods each add an additional penalty term on the coefficients \(\beta\). The LASSO adds the sum of the absolute values of the parameters \(\beta\) subject to a tunable weight, \(\lambda\). The effect of the penalty on the coefficients is to optimize them toward zero. For the LASSO, some coefficients are optimized all the way to zero. Geometric and analytic explanations for this are readily available, with one available at https://en.wikipedia.org/wiki/Lasso_(statistics).

\[ \underset{\beta}{\arg\min} \sum_{n}(y-X\beta)^{2}+ \lambda\sum_{k}|\beta_{k}|\]

The ridge penalty adds the sum of the squares of the parameters \(\beta\), subject to a tunable weight, \(\alpha\).

\[ \underset{\beta}{\arg\min} \sum_{n}(y-X\beta)^{2}+ \alpha\sum_{k}\beta_{k}^{2}\]

The effect of this sort of penalty is still to optimize toward zero. However, unlike the LASSO, they are never driven to zero. Also, a squared-error penalty tends to assymetrically reward small deviations and punish large deviations.

In both cases, for special \(\lambda\) or \(\alpha\), the minimizers of these expressions correspond to the Bayesian maximum a posteriori (MAP) estimators for specific prior distributions for \(\beta\). In the ridge case, the Bayesian prior is the normal distribution with mean 0 and covariance \(\tau^{2} I_{k x k}\) for some assumed \(\tau^{2}\). For the LASSO, the Bayesian prior is the double-exponential or Laplace distribution with mean 0 and parameter \(\tau\). In either case, it can be shown that if \(\sigma^{2}\) and \(\tau\) are known, the penalizing weights have unique solutions equal to the k factor of Bhlmann credibility. In practice, the prior variances are unknown, and the penalizing weights must be tuned. The resulting optimal \(\beta\) is also credible from a Bayesian perspective.

It is also possible to combine the LASSO and ridge. When combined in the glmnet package, the \(\alpha\) parameter is intended as a mixing parameter between the two penalties, and \(\lambda\) controls the overall strength of the penalty.

\[ \underset{\beta}{\arg\min} \sum_{n}(y-X\beta)^{2}+ \lambda(\frac{1-\alpha}{2} \sum_{k}\beta_{k}^{2} + \alpha\sum_{k}|\beta_{k}|)\]

6.5 Interpretation for Elastic Net

Elastic net GLMs differ from GLMs via the penalization term, \(\lambda\). There are two plots related to this. The first is the plot of \(\lambda\) versus the cross-validation. The second is the plot of \(\lambda\) versus the coefficients of the model. Lower values of \(\lambda\) yield lighter penalization and more non-zero terms. Cross-validation is used to find the value which minimizes the out-of-sample error. These plots illustrate the model choices.

Just like GLMs though, the table of factors can be displayed for each variable and interaction term. Absent here are confidence intervals and standard errors. These are not implemented here. However, penalization yields credible parameter estimates. Thus, the factors can be arguably deemed credible.

7 Term Modeling by Amount

7.1 Excluding PLT

Code
dat.term.noplt <- readRDS("dat.term.noplt.rds")
dat.term.noplt <- dat.term.noplt[Policies_Exposed > 0]

# Force usage of all interactions
# This was a setting used in early experiments.
bUseAllInteractions <- FALSE

# Set up the age and durational breaks to group those columns
AA.brks <- c(-1,17,
             seq(25,115,5))
AA.lbls <- paste0(
  AA.brks[1:(length(AA.brks)-1)]+1,
  "-",
  AA.brks[2:length(AA.brks)]
)

Dur.brks <- c(0,1,2,3,5,10,15,20,30,100)
Dur.lbls <- paste0(
  Dur.brks[1:(length(Dur.brks)-1)]+1,
  "-",
  Dur.brks[2:length(Dur.brks)]
)

dat.term.noplt[,`:=`(
  AA_Grp=cut(Attained_Age,
              breaks=AA.brks,
              labels=AA.lbls),
  Dur_Grp=cut(Duration,
               breaks=Dur.brks,
               labels=Dur.lbls)
)]


# Aside the names of the columns we want to use for the model
pred.cols <- names(dat.term.noplt)[c(1,5:6,15:17)]

# Declare the factor columns and ensure that those columns of
# the data.table are classed accordingly
factor.cols <- c("Sex",
                 "Face_Amount_Band",
                 "SOA_Post_Lvl_Ind",
                 "UW",
                 "AA_Grp",
                 "Dur_Grp")

dat.term.noplt[,
      (factor.cols):=lapply(.SD,factor),
      .SDcols=factor.cols]

# The UW variable sorts alphabetically when it is created.
# We want the sort to follow a lexicographic convention of
# smoker status, number of risk classes, and risk class
UW.levels<-data.table(UW.levels=dat.term.noplt[,levels(UW)])
UW.levels[,c("NS","Pref_Class","NClasses"):=tstrsplit(UW.levels,"/")]
setkeyv(UW.levels,c("NS","NClasses","Pref_Class"))

# Mark rows for the train test split for boosting,
# create a Noise column, and rewrite the
# face amount band labels to the more compact version.
set.seed(traintest.seed)
dat.term.noplt %>%
  mutate(
    IsTraining = (runif(nrow(.)) < training.fraction),
    Noise = rnorm(nrow(.)),
    .before = 1
  ) %>% 
  mutate(Face_Amount_Band=fct_relabel(
    Face_Amount_Band,
    #function(.) sub(":"," -",.,fixed=T)
    function(.) fa.remap[Face_Amount_Band.Old==.,Face_Amount_Band.New]
    ),
    AE_Count=Death_Count/ExpDth_Cnt_VBT2015,
    AE_Amount=Death_Claim_Amount/ExpDth_Amt_VBT2015,
    UW=factor(UW,levels=UW.levels$UW.levels)
  ) -> 
  dat.term.noplt

# Set aside the names of the variables for boosting
gbm.pred.cols <- c("Noise",pred.cols)

Here is a data preview.

Code
dat.term.noplt %>%
  head(10) %>%
  flextable()

IsTraining

Noise

Sex

Smoker_Status

Attained_Age

Duration

Face_Amount_Band

SOA_Post_Lvl_Ind

Number_of_Pfd_Classes

Preferred_Class

Death_Count

Policies_Exposed

ExpDth_Cnt_VBT2015

Death_Claim_Amount

Amount_Exposed

ExpDth_Amt_VBT2015

UW

AA_Grp

Dur_Grp

AE_Count

AE_Amount

TRUE

0.1083568294

F

NS

64

6

25K - <50K

ULT

U

U

0

17.645228

0.05716894

0

468,393.7

1,518.3502

NS/U/U

61-65

6-10

0.0000000

0.0000000

TRUE

-1.3291942329

F

NS

64

6

25K - <50K

NLT

2

2

0

3.208219

0.01068337

0

118,677.0

395.1945

NS/2/2

61-65

6-10

0.0000000

0.0000000

TRUE

-0.6640067577

F

NS

64

6

50K - <100K

WLT

U

U

3

488.605865

1.61422224

180,000

27,738,248.8

91,612.1638

NS/U/U

61-65

6-10

1.8584802

1.9648046

TRUE

1.3605113635

F

NS

64

6

50K - <100K

WLT

4

1

0

196.833091

0.62903674

0

10,295,065.5

32,898.3426

NS/1/4

61-65

6-10

0.0000000

0.0000000

TRUE

2.1257419799

F

NS

64

6

50K - <100K

WLT

3

3

0

247.835381

0.80761878

0

13,608,728.6

44,349.2771

NS/3/3

61-65

6-10

0.0000000

0.0000000

TRUE

-0.0009361084

F

NS

64

6

50K - <100K

WLT

3

2

0

14.466719

0.04732497

0

763,336.0

2,495.2484

NS/2/3

61-65

6-10

0.0000000

0.0000000

FALSE

0.0614656053

F

NS

64

6

100K - <250K

WLT

4

1

8

2,087.237293

6.73676967

1,135,000

265,749,659.5

857,260.6139

NS/1/4

61-65

6-10

1.1875128

1.3239848

TRUE

0.6263000098

F

NS

64

6

100K - <250K

WLT

3

2

2

1,080.443395

3.52259762

340,000

131,023,579.0

427,448.7273

NS/2/3

61-65

6-10

0.5677628

0.7954170

TRUE

-0.9311761277

F

NS

64

6

100K - <250K

WLT

3

1

1

890.999727

2.91249311

100,000

109,272,289.0

357,230.3786

NS/1/3

61-65

6-10

0.3433485

0.2799314

TRUE

-0.6049300509

F

NS

64

6

100K - <250K

WLT

4

3

1

1,302.159170

4.21731010

100,000

158,445,453.8

513,118.1666

NS/3/4

61-65

6-10

0.2371180

0.1948869

7.1.1 Feature Discovery with Catboost

As in the predictive analytics framework, gradient boosted machines as implemented with Catboost are used for exploratory data analysis. The framework uses LightGBM, while Catboost is used here. This was done to take advantage of GPUs.

Code
### Create two pools for training and testing data.
### Note that due to a seeming bug in catboost, we
### convert the binary category of Sex to numeric.

### Then fit a catboost model
dat.term.noplt %>% 
  filter(IsTraining == TRUE) %>% 
  select(all_of(gbm.pred.cols)) %>% 
  mutate(Sex = as.numeric(Sex)) %>%
  catboost.load_pool(
    data=.,
    label = dat.term.noplt[IsTraining==TRUE,AE_Amount],
    weight = dat.term.noplt[IsTraining==TRUE,ExpDth_Amt_VBT2015]) ->
  train_pool_notplt

dat.term.noplt %>% 
  filter(IsTraining == FALSE) %>% 
  select(all_of(gbm.pred.cols)) %>% 
  mutate(Sex = as.numeric(Sex)) %>%
  catboost.load_pool(
    data=.,
    label = dat.term.noplt[IsTraining==FALSE,AE_Amount],
    weight = dat.term.noplt[IsTraining==FALSE,ExpDth_Amt_VBT2015]) ->
  test_pool_notplt

if(bUseCache & file.exists(
  'dat.term.mod.notplt.cbm'
) & !bInvalidateCaches)
{
  cb.model.notplt <- catboost.load_model(model_path = 'dat.term.mod.notplt.cbm')
} else {
  cb.model.notplt <- catboost.train(learn_pool=train_pool_notplt,
                             test_pool = test_pool_notplt,
                             params=fit_params)
  
  catboost.save_model(cb.model.notplt,
                      model_path = 'dat.term.mod.notplt.cbm')
}

7.1.2 Feature Importance

Next is to check for the variables with most variability. Face amount band and underwriting top the list, with importance declining for the others. Note that “importance” is a measure of relative variability and not “significance”.

Code
### Extract the shap values. This implicitly relies on a custom function.
shp.notplt <- shapviz(
  cb.model.notplt,
  X=as.data.frame(dat.term.noplt[IsTraining==TRUE,..gbm.pred.cols] %>% mutate(Noise=runif(nrow(.)),.before = 1)),
  X_pred=train_pool_notplt
)

setDT(shp.notplt$X)
shp.notplt$X[,c("NS","Pref_Class","NClasses"):=tstrsplit(UW,"/")]

sv_importance(shp.notplt) + theme_minimal()

7.1.3 Feature Interactions

The top four interactions are face amount band with each of underwriting and duration, and post-level term indicator with underwriting. We look at only those here. For the sake of thoroughness, the top eight are used for the elastic net.

Note that in this subset, SOA_Post_Lvl_Ind contains the levels “WLT = within level term”, “NLT = not level term”, and “ULT = unknown level term”.

Code
### Extract feature interactions, which are unfortunately indexed by column.
### Thus, they need to be returned to their proper names.

imp.int.notplt <- catboost.get_feature_importance(
  cb.model.notplt,
  train_pool_notplt,
  type="Interaction"
)
mod.ft.map.notplt <- data.table(
  FeatureNames=names(dat.term.noplt[IsTraining==TRUE,..gbm.pred.cols])
)

mod.ft.map.notplt[,ID:=1:nrow(.SD)-1]

imp.int.notplt %>%
  merge(mod.ft.map.notplt,
        by.x="feature1_index",
        by.y="ID") %>%
  merge(mod.ft.map.notplt,
        by.x="feature2_index",
        by.y="ID") %>%
  setnames(
    old=c("FeatureNames.x","FeatureNames.y"),
    new=c("Feature1","Feature2")
  ) %>% 
  mutate(feature1_index=NULL,feature2_index=NULL) %>%
  as.data.table() %>%
  setcolorder(c("Feature1","Feature2","score")) -> 
  imp.int.notplt

imp.int.notplt[order(-score)] %>%
  filter(Feature1 != "Noise") %>%
  flextable() %>%
  colformat_double(j="score",digits = 2)

Feature1

Feature2

score

Face_Amount_Band

UW

14.95

Face_Amount_Band

Dur_Grp

10.17

SOA_Post_Lvl_Ind

UW

5.07

Face_Amount_Band

AA_Grp

4.88

Face_Amount_Band

SOA_Post_Lvl_Ind

4.26

UW

Dur_Grp

3.92

UW

AA_Grp

3.01

Sex

UW

2.37

SOA_Post_Lvl_Ind

Dur_Grp

2.21

Sex

Face_Amount_Band

2.14

AA_Grp

Dur_Grp

1.45

Sex

Dur_Grp

1.18

SOA_Post_Lvl_Ind

AA_Grp

0.99

Sex

SOA_Post_Lvl_Ind

0.66

Sex

AA_Grp

0.33

7.1.4 Feature Interaction Plots

Each of the interactions are discussed below with plots using SHAP values from both sides of the interaction. Focus is on the top three. For each interaction, we look at the SHAP values arising from each side of the interaction. For example, the ridge plots under UW for “Face Amount Band SHAPs” are from the face amount effect, while the same for “UW SHAPs” is for the UW effect. Evidence for an interaction can be in one or both collections of SHAP values.

7.1.4.1 UW

7.1.4.1.1 Face Amount Band SHAPs

While the SHAP values for face amount tend to be stable across preferred classes within a given class system, two phenomena stand out:

  1. The SHAP value distribution is shifted rightward for face amounts 25K - <100K.
  2. The SHAP value distribution for those face amounts exhibits greater spread and also multiple modes, suggesting the presence of multiple distinct cohorts.
Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"Face_Amount_Band"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for non-smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"Face_Amount_Band"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '2') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for non-smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"Face_Amount_Band"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '3') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for non-smokers, 3 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"Face_Amount_Band"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '4') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for non-smokers, 4 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"Face_Amount_Band"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"Face_Amount_Band"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == '2') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"Face_Amount_Band"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for unismokers")) +
  theme_minimal()

7.1.4.1.2 UW SHAPs

There is much to unpack from the UW SHAP values stratified by face amount band.

  1. For 2-, 3-, and 4-class systems, there is a persistent pattern of bimodality in the SHAP values.
  1. For 2-class non-smoker systems, the residual standard class exhibits bimodality and a distinct arc of shifting distributions for lower face amounts.
  2. For 3-class non-smoker systems, the bimodality is present across all three levels.
  3. For 4-class non-smoker systems, the bimodality is evident for the best two preferred classes at face amounts above 100K.
  4. Bimodality is evident in the smoker 2-class as well.
  1. Somewhat less obvious is the gradual shifting to the left or right with changes in face amount band across the plots.
Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"UW"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"UW"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '2') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"UW"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '3') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, 3 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"UW"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '4') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, 4 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"UW"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"UW"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == '2') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"UW"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for unismokers")) +
  theme_minimal()

7.1.4.2 Duration Group

Among face amount SHAPs, there is not a clear interaction by duration. However, an interaction is apparent in the duration SHAPs. Note the gradual shifting rightward for durations 21-30 and leftward for durations 31+ in the right-most panel of the second plot.

7.1.4.2.1 Face Amount Band SHAPs
Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    Dur_Grp)],
    shap=shp.notplt$S[,"Face_Amount_Band"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(Dur_Grp2=fct_collapse(
    Dur_Grp,
    ' 1- 5' = c('1-1','2-2','3-3','4-5'),
    ' 6-20' = c('6-10','11-15','16-20'),
    other_level = '21+'
  )) %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Dur_Grp,fill=Dur_Grp)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,1.5)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = "by duration group") +
  theme_minimal() + facet_wrap(vars(Dur_Grp2))

7.1.4.2.2 Duration Group SHAPs
Code
data.table(
  cbind(
    shp.notplt$X[,.(Face_Amount_Band,
                    Dur_Grp)],
    shap=shp.notplt$S[,"Dur_Grp"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(Dur_Grp2=fct_collapse(
    Dur_Grp,
    ' 1- 5' = c('1-1','2-2','3-3','4-5'),
    ' 6-20' = c('6-10','11-15','16-20'),
    other_level = '21+'
  )) %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Dur_Grp,fill=Dur_Grp)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,1.25)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Dur_Grp"),
          subtitle = "by duration group") +
  theme_minimal() + facet_wrap(vars(Dur_Grp2))

7.1.4.3 SOA PLT Indicator vs UW

There are some differences in the spread of preferred factors across the PLT types, specifically for the NLT subgroup.

7.1.4.3.1 UW SHAPs
Code
data.table(
  cbind(
    shp.notplt$X[,.(SOA_Post_Lvl_Ind,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"UW"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == 'U') %>%
  ggplot(aes(y=SOA_Post_Lvl_Ind, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(SOA_Post_Lvl_Ind,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"UW"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '2') %>%
  ggplot(aes(y=SOA_Post_Lvl_Ind, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(SOA_Post_Lvl_Ind,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"UW"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '3') %>%
  ggplot(aes(y=SOA_Post_Lvl_Ind, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, 3 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(SOA_Post_Lvl_Ind,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"UW"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '4') %>%
  ggplot(aes(y=SOA_Post_Lvl_Ind, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, 4 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(SOA_Post_Lvl_Ind,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"UW"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == 'U') %>%
  ggplot(aes(y=SOA_Post_Lvl_Ind, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(SOA_Post_Lvl_Ind,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"UW"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == '2') %>%
  ggplot(aes(y=SOA_Post_Lvl_Ind, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(SOA_Post_Lvl_Ind,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"UW"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'U') %>%
  ggplot(aes(y=SOA_Post_Lvl_Ind, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for unismokers")) +
  theme_minimal()

7.1.4.3.2 SOA PLT Indicator SHAPs
Code
data.table(
  cbind(
    shp.notplt$X[,.(SOA_Post_Lvl_Ind,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"SOA_Post_Lvl_Ind"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == 'U') %>%
  ggplot(aes(y=SOA_Post_Lvl_Ind, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect SOA_Post_Lvl_Ind"),
          subtitle = paste0("for non-smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(SOA_Post_Lvl_Ind,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"SOA_Post_Lvl_Ind"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '2') %>%
  ggplot(aes(y=SOA_Post_Lvl_Ind, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect SOA_Post_Lvl_Ind"),
          subtitle = paste0("for non-smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(SOA_Post_Lvl_Ind,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"SOA_Post_Lvl_Ind"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '3') %>%
  ggplot(aes(y=SOA_Post_Lvl_Ind, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect SOA_Post_Lvl_Ind"),
          subtitle = paste0("for non-smokers, 3 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(SOA_Post_Lvl_Ind,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"SOA_Post_Lvl_Ind"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '4') %>%
  ggplot(aes(y=SOA_Post_Lvl_Ind, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect SOA_Post_Lvl_Ind"),
          subtitle = paste0("for non-smokers, 4 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(SOA_Post_Lvl_Ind,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"SOA_Post_Lvl_Ind"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == 'U') %>%
  ggplot(aes(y=SOA_Post_Lvl_Ind, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect SOA_Post_Lvl_Ind"),
          subtitle = paste0("for smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(SOA_Post_Lvl_Ind,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"SOA_Post_Lvl_Ind"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == '2') %>%
  ggplot(aes(y=SOA_Post_Lvl_Ind, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect SOA_Post_Lvl_Ind"),
          subtitle = paste0("for smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.notplt$X[,.(SOA_Post_Lvl_Ind,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.notplt$S[,"SOA_Post_Lvl_Ind"],
    response=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',AE_Amount],
    offset=dat.term.noplt[IsTraining==TRUE & SOA_Post_Lvl_Ind != 'PLT',ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'U') %>%
  ggplot(aes(y=SOA_Post_Lvl_Ind, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect SOA_Post_Lvl_Ind"),
          subtitle = paste0("for unismokers")) +
  theme_minimal()

7.1.5 Next Steps in Modeling

The catboost modeling is informative here for exploratory analytics. We next build an elastic net model which includes the main effects plus the interactions identified by catboost. Elastic net modeling will reveal which is truly important in a mortality model.

While the work for supporting this is omitted here, it became clear that separate models should be fit for above and below 100K face amounts. A unified model had difficulty adequately fitting face amounts below 100K.

7.1.6 Elastic Net Models

The following formula is used throughout all non-PLT, term models.

Code
if(bUseAllInteractions) {
  glmnetFormula <- as.formula(
    paste0(
      "~ -1 + (",
      paste(pred.cols,collapse=" + "),
      ")^2"
    )
)
} else {
  glmnetFormula <- as.formula(
    paste(c("~ -1",
            pred.cols,
            imp.int.notplt[order(-score)][Feature1 != "Noise",
                           paste0(Feature1,":",Feature2)]),
          collapse=" + ")
  )
}


print(glmnetFormula)
~-1 + Sex + Face_Amount_Band + SOA_Post_Lvl_Ind + UW + AA_Grp + 
    Dur_Grp + Face_Amount_Band:UW + Face_Amount_Band:Dur_Grp + 
    SOA_Post_Lvl_Ind:UW + Face_Amount_Band:AA_Grp + Face_Amount_Band:SOA_Post_Lvl_Ind + 
    UW:Dur_Grp + UW:AA_Grp + Sex:UW + SOA_Post_Lvl_Ind:Dur_Grp + 
    Sex:Face_Amount_Band + AA_Grp:Dur_Grp + Sex:Dur_Grp + SOA_Post_Lvl_Ind:AA_Grp + 
    Sex:SOA_Post_Lvl_Ind + Sex:AA_Grp

The interactions included in the model are as follows:

Code
data.table(Interaction=imp.int.notplt[order(-score)][Feature1 != "Noise",
                         paste0(Feature1,":",Feature2)]) %>%
  flextable()

Interaction

Face_Amount_Band:UW

Face_Amount_Band:Dur_Grp

SOA_Post_Lvl_Ind:UW

Face_Amount_Band:AA_Grp

Face_Amount_Band:SOA_Post_Lvl_Ind

UW:Dur_Grp

UW:AA_Grp

Sex:UW

SOA_Post_Lvl_Ind:Dur_Grp

Sex:Face_Amount_Band

AA_Grp:Dur_Grp

Sex:Dur_Grp

SOA_Post_Lvl_Ind:AA_Grp

Sex:SOA_Post_Lvl_Ind

Sex:AA_Grp

7.1.6.1 Term Model Under 100K, Excluding PLT

The elastic net model is fitted. Note the use of the sparse option. This enables efficient matrix algebra when using the elastic net routines.

Code
### The prepELData function delivers the items
### needed for fitting the elastic net model.

### Then, fit the model

dat.term.noplt.lt100k <- prepELData(
  formula=glmnetFormula,
  data=dat.term.noplt[
                  Face_Amount_Band %in% c("<10K",
    "10K - <25K",
    "25K - <50K",
    "50K - <100K")
  ],
  predictors = setdiff(pred.cols,"Smoker_Status"),
  response = "AE_Amount",
  weights = "ExpDth_Amt_VBT2015",
  useSparse = T,
  dropunused = T
)


set.seed(cvfit.seed)

if(bUseCache & file.exists(
  'term.model.el.noplt.lt100k.rds'
) & !bInvalidateCaches)
{
  cvfit.term.noplt.lt100k <- readRDS('term.model.el.noplt.lt100k.rds')
} else
{
  cvfit.term.noplt.lt100k <- fitCVGLMNet(
    dat.term.noplt.lt100k,
    nfolds = nFolds
  )
  
  if(bUseCache)
    saveRDS(cvfit.term.noplt.lt100k, 'term.model.el.noplt.lt100k.rds')
}
7.1.6.1.1 Usual plots for Elastic Net Models

When presenting elastic net models, the cross validation plot for \(\lambda\) and the coefficient shrinkage plots are provided.

At the minimum \(\lambda\) of 0.0003212, which minimizes the cross-validation error, the model has 382 parameters.

Code
plot(cvfit.term.noplt.lt100k)

Code
plot(cvfit.term.noplt.lt100k$glmnet.fit,xvar="lambda")

7.1.6.1.2 Factor Table

Their exponentiated coefficients are as follows:

Code
reformatCoefs(cvfit.term.noplt.lt100k, pred.cols)  %>%
  filter(Coef != 0) %>%
  select(Feature1Name,
         Feature1Level,
         Feature2Name,
         Feature2Level,
         Coef) %>%
  mutate(Coef=exp(Coef)) %>%
  flextable() %>%
  set_formatter(
    Coef=function(x) paste0(sprintf("%.01f", 100*x),"%")
  ) %>%
  theme_vanilla()

Feature1Name

Feature1Level

Feature2Name

Feature2Level

Coef

(Intercept)

(Intercept)

178.2%

Sex

F

94.7%

Sex

M

106.8%

Face_Amount_Band

10K - <25K

108.8%

Face_Amount_Band

25K - <50K

99.1%

Face_Amount_Band

50K - <100K

80.6%

SOA_Post_Lvl_Ind

ULT

100.7%

UW

NS/2/2

166.1%

UW

NS/1/3

92.0%

UW

NS/2/3

113.6%

UW

NS/3/3

109.0%

UW

NS/1/4

78.7%

UW

NS/2/4

90.3%

UW

NS/4/4

126.1%

UW

NS/U/U

142.0%

UW

S/2/2

112.0%

UW

S/U/U

112.0%

UW

U/U/U

90.1%

AA_Grp

26-30

140.4%

AA_Grp

31-35

141.3%

AA_Grp

36-40

108.1%

AA_Grp

41-45

109.0%

AA_Grp

46-50

111.8%

AA_Grp

51-55

118.3%

AA_Grp

61-65

93.2%

AA_Grp

71-75

102.1%

AA_Grp

76-80

99.0%

AA_Grp

81-85

98.0%

AA_Grp

86-90

84.3%

AA_Grp

91-95

74.3%

AA_Grp

96-100

90.1%

Dur_Grp

2-2

98.3%

Dur_Grp

4-5

85.6%

Dur_Grp

6-10

80.5%

Dur_Grp

11-15

75.5%

Dur_Grp

16-20

88.7%

Dur_Grp

21-30

78.1%

Dur_Grp

31-100

66.2%

Face_Amount_Band

10K - <25K

UW

NS/2/2

88.8%

Face_Amount_Band

50K - <100K

UW

NS/2/2

112.6%

Face_Amount_Band

25K - <50K

UW

NS/2/3

110.6%

Face_Amount_Band

50K - <100K

UW

NS/3/3

114.9%

Face_Amount_Band

50K - <100K

UW

NS/1/4

89.3%

Face_Amount_Band

50K - <100K

UW

NS/2/4

95.9%

Face_Amount_Band

50K - <100K

UW

NS/4/4

111.2%

Face_Amount_Band

10K - <25K

UW

NS/U/U

100.1%

Face_Amount_Band

25K - <50K

UW

NS/U/U

92.3%

Face_Amount_Band

50K - <100K

UW

NS/U/U

99.0%

Face_Amount_Band

10K - <25K

UW

S/U/U

95.9%

Face_Amount_Band

50K - <100K

UW

S/U/U

113.6%

Face_Amount_Band

50K - <100K

UW

U/U/U

109.8%

Face_Amount_Band

25K - <50K

Dur_Grp

2-2

90.9%

Face_Amount_Band

50K - <100K

Dur_Grp

2-2

83.5%

Face_Amount_Band

10K - <25K

Dur_Grp

3-3

100.1%

Face_Amount_Band

25K - <50K

Dur_Grp

3-3

94.0%

Face_Amount_Band

50K - <100K

Dur_Grp

3-3

99.5%

Face_Amount_Band

10K - <25K

Dur_Grp

4-5

93.1%

Face_Amount_Band

25K - <50K

Dur_Grp

4-5

93.5%

Face_Amount_Band

10K - <25K

Dur_Grp

6-10

100.0%

Face_Amount_Band

25K - <50K

Dur_Grp

6-10

99.9%

Face_Amount_Band

50K - <100K

Dur_Grp

6-10

96.3%

Face_Amount_Band

25K - <50K

Dur_Grp

11-15

100.4%

Face_Amount_Band

50K - <100K

Dur_Grp

11-15

94.6%

Face_Amount_Band

10K - <25K

Dur_Grp

16-20

79.4%

Face_Amount_Band

25K - <50K

Dur_Grp

16-20

83.1%

Face_Amount_Band

50K - <100K

Dur_Grp

16-20

72.3%

Face_Amount_Band

10K - <25K

Dur_Grp

21-30

84.6%

Face_Amount_Band

50K - <100K

Dur_Grp

21-30

98.3%

Face_Amount_Band

25K - <50K

Dur_Grp

31-100

114.4%

Face_Amount_Band

50K - <100K

Dur_Grp

31-100

115.7%

SOA_Post_Lvl_Ind

NLT

UW

NS/2/2

77.4%

SOA_Post_Lvl_Ind

ULT

UW

NS/2/2

106.0%

SOA_Post_Lvl_Ind

ULT

UW

NS/1/3

110.6%

SOA_Post_Lvl_Ind

NLT

UW

NS/2/3

95.4%

SOA_Post_Lvl_Ind

NLT

UW

NS/3/3

102.0%

SOA_Post_Lvl_Ind

NLT

UW

NS/U/U

102.9%

SOA_Post_Lvl_Ind

ULT

UW

NS/U/U

99.6%

SOA_Post_Lvl_Ind

ULT

UW

S/1/2

106.1%

SOA_Post_Lvl_Ind

NLT

UW

S/U/U

97.0%

SOA_Post_Lvl_Ind

ULT

UW

S/U/U

98.0%

SOA_Post_Lvl_Ind

NLT

UW

U/U/U

110.4%

SOA_Post_Lvl_Ind

ULT

UW

U/U/U

103.5%

Face_Amount_Band

25K - <50K

AA_Grp

18-25

91.0%

Face_Amount_Band

25K - <50K

AA_Grp

26-30

112.9%

Face_Amount_Band

25K - <50K

AA_Grp

31-35

101.2%

Face_Amount_Band

50K - <100K

AA_Grp

36-40

105.2%

Face_Amount_Band

25K - <50K

AA_Grp

41-45

107.9%

Face_Amount_Band

50K - <100K

AA_Grp

46-50

95.6%

Face_Amount_Band

25K - <50K

AA_Grp

51-55

101.3%

Face_Amount_Band

50K - <100K

AA_Grp

51-55

89.6%

Face_Amount_Band

10K - <25K

AA_Grp

56-60

103.9%

Face_Amount_Band

50K - <100K

AA_Grp

56-60

96.3%

Face_Amount_Band

25K - <50K

AA_Grp

61-65

98.1%

Face_Amount_Band

50K - <100K

AA_Grp

61-65

95.5%

Face_Amount_Band

10K - <25K

AA_Grp

66-70

100.7%

Face_Amount_Band

25K - <50K

AA_Grp

66-70

93.5%

Face_Amount_Band

50K - <100K

AA_Grp

66-70

95.4%

Face_Amount_Band

25K - <50K

AA_Grp

71-75

100.3%

Face_Amount_Band

50K - <100K

AA_Grp

71-75

104.4%

Face_Amount_Band

50K - <100K

AA_Grp

76-80

114.1%

Face_Amount_Band

25K - <50K

AA_Grp

86-90

89.9%

Face_Amount_Band

50K - <100K

AA_Grp

86-90

101.8%

Face_Amount_Band

50K - <100K

AA_Grp

91-95

97.5%

Face_Amount_Band

10K - <25K

SOA_Post_Lvl_Ind

NLT

82.0%

Face_Amount_Band

25K - <50K

SOA_Post_Lvl_Ind

NLT

70.4%

Face_Amount_Band

50K - <100K

SOA_Post_Lvl_Ind

NLT

78.3%

Face_Amount_Band

10K - <25K

SOA_Post_Lvl_Ind

ULT

118.4%

Face_Amount_Band

25K - <50K

SOA_Post_Lvl_Ind

ULT

118.3%

Face_Amount_Band

50K - <100K

SOA_Post_Lvl_Ind

ULT

126.7%

UW

NS/2/2

Dur_Grp

2-2

104.5%

UW

NS/U/U

Dur_Grp

2-2

109.3%

UW

NS/2/2

Dur_Grp

3-3

100.3%

UW

NS/2/4

Dur_Grp

3-3

103.8%

UW

NS/3/4

Dur_Grp

3-3

86.9%

UW

NS/U/U

Dur_Grp

3-3

98.2%

UW

S/1/2

Dur_Grp

3-3

102.9%

UW

S/2/2

Dur_Grp

3-3

103.3%

UW

NS/2/2

Dur_Grp

4-5

102.3%

UW

NS/3/3

Dur_Grp

4-5

108.5%

UW

S/1/2

Dur_Grp

4-5

87.1%

UW

S/U/U

Dur_Grp

4-5

98.4%

UW

NS/2/2

Dur_Grp

6-10

93.6%

UW

NS/3/3

Dur_Grp

6-10

112.5%

UW

NS/1/4

Dur_Grp

6-10

109.3%

UW

NS/4/4

Dur_Grp

6-10

111.1%

UW

NS/U/U

Dur_Grp

6-10

98.4%

UW

S/1/2

Dur_Grp

6-10

93.7%

UW

S/U/U

Dur_Grp

6-10

93.6%

UW

NS/2/2

Dur_Grp

11-15

91.2%

UW

NS/3/3

Dur_Grp

11-15

105.2%

UW

NS/2/4

Dur_Grp

11-15

105.5%

UW

NS/3/4

Dur_Grp

11-15

110.0%

UW

NS/4/4

Dur_Grp

11-15

103.7%

UW

NS/U/U

Dur_Grp

11-15

87.6%

UW

S/2/2

Dur_Grp

11-15

107.9%

UW

S/U/U

Dur_Grp

11-15

108.9%

UW

U/U/U

Dur_Grp

11-15

110.0%

UW

NS/1/3

Dur_Grp

16-20

88.1%

UW

NS/2/3

Dur_Grp

16-20

111.4%

UW

NS/3/3

Dur_Grp

16-20

111.2%

UW

NS/1/4

Dur_Grp

16-20

96.6%

UW

NS/3/4

Dur_Grp

16-20

98.8%

UW

NS/U/U

Dur_Grp

16-20

85.4%

UW

S/1/2

Dur_Grp

16-20

115.3%

UW

S/2/2

Dur_Grp

16-20

121.3%

UW

S/U/U

Dur_Grp

16-20

118.4%

UW

NS/2/2

Dur_Grp

21-30

68.7%

UW

NS/3/3

Dur_Grp

21-30

88.5%

UW

NS/U/U

Dur_Grp

21-30

72.8%

UW

S/2/2

Dur_Grp

21-30

101.0%

UW

S/U/U

Dur_Grp

21-30

104.0%

UW

NS/2/2

Dur_Grp

31-100

64.8%

UW

NS/U/U

Dur_Grp

31-100

68.1%

UW

S/U/U

Dur_Grp

31-100

79.5%

UW

U/U/U

Dur_Grp

31-100

98.3%

UW

NS/2/2

AA_Grp

18-25

83.8%

UW

NS/U/U

AA_Grp

18-25

69.1%

UW

S/U/U

AA_Grp

18-25

110.0%

UW

NS/4/4

AA_Grp

26-30

108.5%

UW

NS/U/U

AA_Grp

26-30

77.5%

UW

S/U/U

AA_Grp

26-30

120.5%

UW

U/U/U

AA_Grp

26-30

101.2%

UW

NS/2/2

AA_Grp

31-35

116.6%

UW

NS/1/4

AA_Grp

31-35

119.1%

UW

NS/U/U

AA_Grp

31-35

87.5%

UW

S/U/U

AA_Grp

31-35

111.9%

UW

NS/2/2

AA_Grp

36-40

124.8%

UW

NS/3/3

AA_Grp

36-40

114.7%

UW

S/2/2

AA_Grp

36-40

99.6%

UW

S/U/U

AA_Grp

36-40

98.6%

UW

NS/3/3

AA_Grp

41-45

127.2%

UW

NS/U/U

AA_Grp

41-45

115.4%

UW

S/U/U

AA_Grp

41-45

85.4%

UW

NS/2/2

AA_Grp

46-50

96.1%

UW

NS/4/4

AA_Grp

46-50

100.6%

UW

NS/U/U

AA_Grp

46-50

102.3%

UW

S/1/2

AA_Grp

46-50

95.0%

UW

S/2/2

AA_Grp

46-50

97.4%

UW

U/U/U

AA_Grp

46-50

114.5%

UW

NS/2/2

AA_Grp

51-55

90.3%

UW

NS/2/3

AA_Grp

51-55

111.2%

UW

NS/3/3

AA_Grp

51-55

108.1%

UW

NS/3/4

AA_Grp

51-55

101.4%

UW

S/1/2

AA_Grp

51-55

91.5%

UW

S/2/2

AA_Grp

51-55

106.1%

UW

S/U/U

AA_Grp

51-55

86.3%

UW

NS/2/2

AA_Grp

56-60

99.5%

UW

NS/2/3

AA_Grp

56-60

109.3%

UW

NS/3/3

AA_Grp

56-60

102.7%

UW

NS/2/4

AA_Grp

56-60

93.5%

UW

NS/3/4

AA_Grp

56-60

74.2%

UW

NS/4/4

AA_Grp

56-60

100.2%

UW

NS/U/U

AA_Grp

56-60

110.2%

UW

S/2/2

AA_Grp

56-60

89.7%

UW

S/U/U

AA_Grp

56-60

102.5%

UW

U/U/U

AA_Grp

56-60

107.0%

UW

NS/2/2

AA_Grp

61-65

103.7%

UW

NS/1/3

AA_Grp

61-65

98.5%

UW

NS/2/3

AA_Grp

61-65

91.9%

UW

NS/1/4

AA_Grp

61-65

97.3%

UW

NS/2/4

AA_Grp

61-65

97.1%

UW

NS/3/4

AA_Grp

61-65

102.2%

UW

NS/4/4

AA_Grp

61-65

112.0%

UW

NS/U/U

AA_Grp

61-65

105.4%

UW

S/U/U

AA_Grp

61-65

98.0%

UW

NS/2/2

AA_Grp

66-70

113.3%

UW

NS/1/3

AA_Grp

66-70

98.5%

UW

NS/2/3

AA_Grp

66-70

101.1%

UW

NS/4/4

AA_Grp

66-70

94.5%

UW

NS/U/U

AA_Grp

66-70

112.7%

UW

S/1/2

AA_Grp

66-70

107.9%

UW

S/U/U

AA_Grp

66-70

106.4%

UW

U/U/U

AA_Grp

66-70

116.9%

UW

NS/2/2

AA_Grp

71-75

104.6%

UW

NS/2/3

AA_Grp

71-75

93.3%

UW

NS/3/3

AA_Grp

71-75

91.2%

UW

NS/2/4

AA_Grp

71-75

110.8%

UW

NS/4/4

AA_Grp

71-75

78.5%

UW

NS/U/U

AA_Grp

71-75

103.3%

UW

S/1/2

AA_Grp

71-75

100.0%

UW

S/U/U

AA_Grp

71-75

101.0%

UW

NS/2/2

AA_Grp

76-80

104.0%

UW

NS/3/3

AA_Grp

76-80

90.8%

UW

NS/2/4

AA_Grp

76-80

99.0%

UW

NS/4/4

AA_Grp

76-80

76.0%

UW

NS/U/U

AA_Grp

76-80

102.7%

UW

S/1/2

AA_Grp

76-80

112.8%

UW

S/2/2

AA_Grp

76-80

115.7%

UW

S/U/U

AA_Grp

76-80

103.1%

UW

U/U/U

AA_Grp

76-80

87.0%

UW

NS/1/3

AA_Grp

81-85

119.1%

UW

NS/2/3

AA_Grp

81-85

98.3%

UW

NS/1/4

AA_Grp

81-85

137.4%

UW

NS/2/4

AA_Grp

81-85

116.1%

UW

NS/4/4

AA_Grp

81-85

76.3%

UW

NS/U/U

AA_Grp

81-85

97.1%

UW

U/U/U

AA_Grp

81-85

76.1%

UW

NS/2/2

AA_Grp

86-90

84.6%

UW

U/U/U

AA_Grp

86-90

80.2%

UW

NS/U/U

AA_Grp

91-95

107.0%

Sex

M

UW

NS/2/2

105.4%

Sex

M

UW

NS/1/3

99.8%

Sex

M

UW

NS/2/3

95.4%

Sex

M

UW

NS/3/3

97.3%

Sex

M

UW

NS/2/4

102.3%

Sex

M

UW

NS/3/4

108.2%

Sex

M

UW

NS/4/4

100.4%

Sex

M

UW

S/1/2

89.6%

Sex

M

UW

S/2/2

96.9%

Sex

M

UW

S/U/U

95.4%

SOA_Post_Lvl_Ind

NLT

Dur_Grp

2-2

170.1%

SOA_Post_Lvl_Ind

NLT

Dur_Grp

3-3

89.2%

SOA_Post_Lvl_Ind

ULT

Dur_Grp

3-3

88.5%

SOA_Post_Lvl_Ind

ULT

Dur_Grp

4-5

91.9%

SOA_Post_Lvl_Ind

NLT

Dur_Grp

6-10

106.0%

SOA_Post_Lvl_Ind

ULT

Dur_Grp

6-10

85.0%

SOA_Post_Lvl_Ind

NLT

Dur_Grp

11-15

99.4%

SOA_Post_Lvl_Ind

ULT

Dur_Grp

11-15

73.0%

SOA_Post_Lvl_Ind

ULT

Dur_Grp

16-20

71.7%

SOA_Post_Lvl_Ind

NLT

Dur_Grp

21-30

100.0%

SOA_Post_Lvl_Ind

ULT

Dur_Grp

21-30

81.7%

SOA_Post_Lvl_Ind

ULT

Dur_Grp

31-100

87.5%

Sex

M

Face_Amount_Band

10K - <25K

99.9%

Sex

M

Face_Amount_Band

25K - <50K

104.3%

Sex

M

Face_Amount_Band

50K - <100K

102.0%

AA_Grp

18-25

Dur_Grp

2-2

87.5%

AA_Grp

31-35

Dur_Grp

2-2

94.8%

AA_Grp

36-40

Dur_Grp

2-2

106.9%

AA_Grp

41-45

Dur_Grp

2-2

96.8%

AA_Grp

46-50

Dur_Grp

2-2

106.0%

AA_Grp

51-55

Dur_Grp

2-2

98.5%

AA_Grp

56-60

Dur_Grp

2-2

94.3%

AA_Grp

61-65

Dur_Grp

2-2

91.4%

AA_Grp

71-75

Dur_Grp

2-2

114.7%

AA_Grp

76-80

Dur_Grp

2-2

134.7%

AA_Grp

81-85

Dur_Grp

2-2

151.6%

AA_Grp

18-25

Dur_Grp

3-3

75.3%

AA_Grp

26-30

Dur_Grp

3-3

101.4%

AA_Grp

31-35

Dur_Grp

3-3

100.6%

AA_Grp

36-40

Dur_Grp

3-3

85.6%

AA_Grp

41-45

Dur_Grp

3-3

100.8%

AA_Grp

46-50

Dur_Grp

3-3

107.5%

AA_Grp

56-60

Dur_Grp

3-3

99.1%

AA_Grp

66-70

Dur_Grp

3-3

98.5%

AA_Grp

71-75

Dur_Grp

3-3

98.6%

AA_Grp

81-85

Dur_Grp

3-3

123.6%

AA_Grp

26-30

Dur_Grp

4-5

132.5%

AA_Grp

36-40

Dur_Grp

4-5

86.8%

AA_Grp

51-55

Dur_Grp

4-5

102.4%

AA_Grp

61-65

Dur_Grp

4-5

97.3%

AA_Grp

66-70

Dur_Grp

4-5

97.8%

AA_Grp

71-75

Dur_Grp

4-5

101.8%

AA_Grp

18-25

Dur_Grp

6-10

102.1%

AA_Grp

26-30

Dur_Grp

6-10

115.5%

AA_Grp

31-35

Dur_Grp

6-10

119.8%

AA_Grp

36-40

Dur_Grp

6-10

112.9%

AA_Grp

41-45

Dur_Grp

6-10

96.6%

AA_Grp

46-50

Dur_Grp

6-10

118.0%

AA_Grp

56-60

Dur_Grp

6-10

108.8%

AA_Grp

61-65

Dur_Grp

6-10

97.0%

AA_Grp

66-70

Dur_Grp

6-10

89.5%

AA_Grp

71-75

Dur_Grp

6-10

91.0%

AA_Grp

76-80

Dur_Grp

6-10

87.4%

AA_Grp

81-85

Dur_Grp

6-10

84.2%

AA_Grp

18-25

Dur_Grp

11-15

97.6%

AA_Grp

36-40

Dur_Grp

11-15

109.2%

AA_Grp

41-45

Dur_Grp

11-15

103.4%

AA_Grp

46-50

Dur_Grp

11-15

99.0%

AA_Grp

56-60

Dur_Grp

11-15

103.4%

AA_Grp

61-65

Dur_Grp

11-15

108.0%

AA_Grp

71-75

Dur_Grp

11-15

91.3%

AA_Grp

76-80

Dur_Grp

11-15

96.3%

AA_Grp

81-85

Dur_Grp

11-15

90.3%

AA_Grp

36-40

Dur_Grp

16-20

86.2%

AA_Grp

41-45

Dur_Grp

16-20

120.8%

AA_Grp

51-55

Dur_Grp

16-20

112.0%

AA_Grp

56-60

Dur_Grp

16-20

110.4%

AA_Grp

61-65

Dur_Grp

16-20

109.7%

AA_Grp

66-70

Dur_Grp

16-20

105.0%

AA_Grp

71-75

Dur_Grp

16-20

89.2%

AA_Grp

76-80

Dur_Grp

16-20

88.8%

AA_Grp

81-85

Dur_Grp

16-20

88.4%

AA_Grp

86-90

Dur_Grp

16-20

95.2%

AA_Grp

41-45

Dur_Grp

21-30

125.2%

AA_Grp

46-50

Dur_Grp

21-30

106.1%

AA_Grp

51-55

Dur_Grp

21-30

90.5%

AA_Grp

56-60

Dur_Grp

21-30

96.3%

AA_Grp

66-70

Dur_Grp

21-30

102.5%

AA_Grp

76-80

Dur_Grp

21-30

97.1%

AA_Grp

91-95

Dur_Grp

21-30

98.8%

AA_Grp

46-50

Dur_Grp

31-100

134.0%

AA_Grp

51-55

Dur_Grp

31-100

102.2%

AA_Grp

56-60

Dur_Grp

31-100

90.6%

AA_Grp

61-65

Dur_Grp

31-100

93.7%

AA_Grp

71-75

Dur_Grp

31-100

96.0%

AA_Grp

76-80

Dur_Grp

31-100

96.9%

AA_Grp

81-85

Dur_Grp

31-100

97.7%

AA_Grp

86-90

Dur_Grp

31-100

100.0%

Sex

M

Dur_Grp

2-2

99.9%

Sex

M

Dur_Grp

4-5

103.6%

Sex

M

Dur_Grp

6-10

104.0%

Sex

M

Dur_Grp

16-20

100.6%

Sex

M

Dur_Grp

21-30

99.7%

Sex

M

Dur_Grp

31-100

102.5%

SOA_Post_Lvl_Ind

ULT

AA_Grp

18-25

115.9%

SOA_Post_Lvl_Ind

ULT

AA_Grp

26-30

104.5%

SOA_Post_Lvl_Ind

ULT

AA_Grp

31-35

81.1%

SOA_Post_Lvl_Ind

ULT

AA_Grp

36-40

93.3%

SOA_Post_Lvl_Ind

NLT

AA_Grp

41-45

89.3%

SOA_Post_Lvl_Ind

ULT

AA_Grp

41-45

86.3%

SOA_Post_Lvl_Ind

NLT

AA_Grp

46-50

94.5%

SOA_Post_Lvl_Ind

ULT

AA_Grp

46-50

103.0%

SOA_Post_Lvl_Ind

NLT

AA_Grp

51-55

106.1%

SOA_Post_Lvl_Ind

ULT

AA_Grp

51-55

99.5%

SOA_Post_Lvl_Ind

NLT

AA_Grp

61-65

112.5%

SOA_Post_Lvl_Ind

ULT

AA_Grp

61-65

113.2%

SOA_Post_Lvl_Ind

ULT

AA_Grp

66-70

106.6%

SOA_Post_Lvl_Ind

NLT

AA_Grp

71-75

111.0%

SOA_Post_Lvl_Ind

ULT

AA_Grp

71-75

127.0%

SOA_Post_Lvl_Ind

NLT

AA_Grp

76-80

104.7%

SOA_Post_Lvl_Ind

ULT

AA_Grp

76-80

127.0%

SOA_Post_Lvl_Ind

ULT

AA_Grp

81-85

131.5%

SOA_Post_Lvl_Ind

ULT

AA_Grp

86-90

131.9%

SOA_Post_Lvl_Ind

ULT

AA_Grp

91-95

94.4%

SOA_Post_Lvl_Ind

ULT

AA_Grp

96-100

90.3%

Sex

M

SOA_Post_Lvl_Ind

NLT

90.6%

Sex

M

SOA_Post_Lvl_Ind

ULT

102.5%

Sex

M

AA_Grp

18-25

111.9%

Sex

M

AA_Grp

26-30

105.6%

Sex

M

AA_Grp

31-35

132.2%

Sex

M

AA_Grp

36-40

137.3%

Sex

M

AA_Grp

41-45

105.5%

Sex

M

AA_Grp

46-50

104.4%

Sex

M

AA_Grp

51-55

113.1%

Sex

M

AA_Grp

56-60

102.6%

Sex

M

AA_Grp

61-65

104.7%

Sex

M

AA_Grp

71-75

90.8%

Sex

M

AA_Grp

76-80

91.0%

Sex

M

AA_Grp

81-85

93.8%

Sex

M

AA_Grp

86-90

93.4%

Sex

M

AA_Grp

91-95

97.5%

Sex

M

AA_Grp

96-100

98.3%

Note that interaction terms must be considered together. For example, UW interacts with Face Amount Band, so these should be considered in combination. The slate of UW without a second feature apply to the base level of Face Amount Band (and any other factor with which UW interacts).

Since penalization is roughly equivalent to Bayesian credibility using a combination of Gaussian and Laplace priors, these factors are arguably credible. This is true even when dealing with factors with small effect sizes, such as the standalone factor for Duration 3.

Plots and tables of effects are much more digestible. As it happens, all but Sex are involved in feature interactions. Those factors can be read from the table above.

In the plots and tables that follow, we fix all of the other variables at their middle values when extracting final factors.

We generate some supporting tables: a factor grid for attaching to the experience, and a list of interactions present in the model

Code
### The following code creates a grid of factors,
### saves the grid to an Excel file, and
### saves a listing of interactions present in the model.
dat.term.noplt[(Face_Amount_Band %in% c("<10K",
    "10K - <25K",
    "25K - <50K",
    "50K - <100K")),..pred.cols] %>%
  mutate(Face_Amount_Band=fct_drop(Face_Amount_Band)) %>%
  lapply(levels) %>%
  expand.grid() -> 
  dat.term.noplt.lt100k.grid 

dat.term.noplt.lt100k.grid %>%
  model.Matrix(
    object=glmnetFormula,
    data=.,
    sparse=T
  ) %>%
  predict(
    cvfit.term.noplt.lt100k,
    newx=.,
    s="lambda.min"
  ) %>% 
  as.vector() ->
  newCoef

dat.term.noplt.lt100k.grid %>%
  add_column(
    Factor=exp(newCoef)
  )  %>% 
  setDT() -> 
  dat.term.noplt.lt100k.grid 

write.xlsx(dat.term.noplt.lt100k.grid,
           file="dat.term.noplt.lt100k.grid.xlsx")

reformatCoefs(cvfit.term.noplt.lt100k, pred.cols) %>%
  filter(Coef != 0 & !is.na(Feature2Name)) %>% 
  select(Feature1Name,Feature2Name) %>% 
  distinct() %>%
  as.list() %>%
  purrr::list_transpose() ->
  term.no.plt.lt100k.int.list
7.1.6.1.3 Plots of Terms

Below are plots of the 2-way interaction terms, with external factors fixed at their middle values.

Some of the effects are subtle and can be confirmed by reviewing the factor table.

  1. Face Amount Band x UW: While factors decline with increasing face amount, the slope of the decline is qualitatively different. Moreover, the first two bands do not always obey this pattern across UW classes.
  2. Dur Group x Face Amount Band: While factors tend to decline with duration, the slope of the decline changes with face amount.
  3. SOA Post Level Ind vs UW: As foreshadowed in the review of SHAP values, there are differences in the spread of preferred factors by post level indicator, particularly in the 2-class systems.
  4. Attained Age x Face Amount Band: there are some differences at the younger attained ages and lowest band.
  5. Face Amount Band x Post Level Indicator: Factors tend to decrease, although the shapes of that decrease differ by post level indicator.
  6. Duration x UW: There is a pattern in the 2-class and 3-class residual standard non-smoker duration patterns, with a pronounced elevation through duration 20.
  7. Attained age x UW: To the extent an interaction is being detected, it is mostly at the younger attained ages.
  8. Sex x UW: The interaction appears most prominent with NS/2/2 which shares a wider spread.
  9. Duration x Post Level Indicator: ULT and NLT deviate from WLT in the earlier durations.
  10. Face Amount Band x Sex: no obvious interaction.
  11. Attained age x Duration: There are some interactions in the earliest durations for both the youngest and the middle attained ages.
  12. Duration x Sex: Nothing stands out. A review of the factor table shows subtle shifts.
  13. Attained Age x Post Level Indicator: Qualitatively, the slope seems different for ULT here.
  14. Post-Level Indicator x Sex: The interaction here appears to be with NLT and males.
  15. Attained age x Sex: There appears to be a different attained slope, most prominent amount younger males.
7.1.6.1.4 Tables of Terms

Below are tables of the 2-way interaction terms, with external factors fixed at their middle values.

Face_Amount_Band

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

<10K

130.8%

230.7%

118.6%

136.6%

154.7%

100.1%

114.7%

133.7%

184.6%

195.7%

113.9%

146.5%

141.2%

117.8%

10K - <25K

132.4%

207.2%

120.0%

138.2%

156.5%

101.3%

116.0%

135.3%

186.8%

198.3%

115.3%

148.2%

136.9%

119.2%

25K - <50K

119.0%

209.8%

107.8%

137.4%

140.7%

91.1%

104.3%

121.6%

167.9%

164.3%

103.6%

133.2%

128.4%

107.2%

50K - <100K

100.7%

200.0%

91.3%

105.1%

136.8%

68.8%

84.7%

102.9%

158.0%

149.1%

87.7%

112.7%

123.4%

99.6%

Dur_Grp

<10K

10K - <25K

25K - <50K

50K - <100K

1-1

137.8%

149.8%

134.0%

101.7%

2-2

123.9%

134.7%

109.5%

76.3%

3-3

143.0%

155.6%

130.6%

105.0%

4-5

114.7%

116.0%

104.3%

84.7%

6-10

107.6%

117.0%

104.5%

76.5%

11-15

118.5%

128.9%

115.7%

82.7%

16-20

134.1%

115.7%

108.3%

71.6%

21-30

107.6%

99.0%

104.7%

78.1%

31-100

85.5%

93.0%

95.1%

73.1%

SOA_Post_Lvl_Ind

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

WLT

132.4%

207.2%

120.0%

138.2%

156.5%

101.3%

116.0%

135.3%

186.8%

198.3%

115.3%

148.2%

136.9%

119.2%

NLT

122.1%

147.9%

110.7%

121.6%

147.3%

93.5%

107.0%

124.8%

172.4%

188.2%

106.3%

136.7%

122.5%

121.4%

ULT

164.1%

272.2%

164.5%

171.4%

194.0%

125.6%

143.9%

167.8%

231.7%

244.9%

151.6%

183.8%

166.3%

153.0%

AA_Grp

<10K

10K - <25K

25K - <50K

50K - <100K

0-17

130.3%

131.8%

120.8%

100.7%

18-25

130.3%

131.8%

109.9%

100.7%

26-30

242.2%

245.1%

253.5%

187.3%

31-35

184.0%

186.3%

172.6%

142.3%

36-40

122.2%

123.6%

113.3%

99.3%

41-45

142.0%

143.7%

142.1%

109.8%

46-50

145.6%

147.3%

135.0%

107.6%

51-55

157.8%

159.7%

148.2%

109.3%

56-60

121.9%

128.1%

113.0%

90.7%

61-65

114.7%

116.0%

104.3%

84.7%

66-70

127.4%

129.9%

110.5%

94.0%

71-75

150.1%

151.9%

139.6%

121.2%

76-80

127.6%

129.2%

118.3%

112.6%

81-85

148.1%

149.9%

137.3%

114.5%

86-90

109.8%

111.1%

91.5%

86.4%

91-95

96.8%

97.9%

89.7%

73.0%

96-100

117.4%

118.8%

108.9%

90.8%

101-105

130.3%

131.8%

120.8%

100.7%

106-110

130.3%

131.8%

120.8%

100.7%

111-115

130.3%

131.8%

120.8%

100.7%

Face_Amount_Band

WLT

NLT

ULT

<10K

114.7%

129.0%

120.1%

10K - <25K

116.0%

107.0%

143.9%

25K - <50K

104.3%

82.6%

129.2%

50K - <100K

84.7%

74.6%

112.3%

Dur_Grp

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

1-1

170.9%

261.3%

154.9%

178.5%

186.3%

130.8%

149.8%

174.7%

241.3%

256.0%

170.9%

191.4%

179.7%

154.0%

2-2

153.6%

245.6%

139.3%

160.4%

167.4%

117.6%

134.7%

157.0%

216.9%

251.6%

153.6%

172.0%

161.5%

138.4%

3-3

171.1%

262.3%

155.1%

178.7%

186.5%

131.0%

155.6%

152.1%

241.5%

251.7%

176.1%

197.9%

179.9%

154.1%

4-5

132.4%

207.2%

120.0%

138.2%

156.5%

101.3%

116.0%

135.3%

186.8%

198.3%

115.3%

148.2%

136.9%

119.2%

6-10

133.4%

190.9%

120.9%

139.3%

163.5%

111.6%

117.0%

136.4%

209.2%

196.7%

125.0%

149.4%

131.2%

120.2%

11-15

139.4%

194.5%

126.3%

145.5%

159.7%

106.7%

128.9%

156.7%

204.1%

182.9%

139.4%

168.4%

159.6%

138.1%

16-20

132.0%

201.9%

105.4%

153.6%

160.0%

97.6%

115.7%

133.4%

186.4%

168.9%

152.3%

179.4%

164.4%

118.9%

21-30

113.0%

118.8%

102.4%

118.0%

108.9%

86.5%

99.0%

115.5%

159.5%

123.2%

113.0%

127.8%

123.6%

101.8%

31-100

106.1%

105.2%

96.2%

110.8%

115.6%

81.2%

93.0%

108.5%

149.8%

108.3%

106.1%

118.8%

88.7%

94.0%

AA_Grp

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

0-17

146.0%

220.3%

134.4%

166.0%

172.7%

114.9%

131.8%

146.0%

184.1%

207.6%

127.2%

163.5%

154.2%

131.5%

18-25

146.0%

184.6%

134.4%

166.0%

172.7%

114.9%

131.8%

146.0%

184.1%

143.5%

127.2%

163.5%

169.6%

131.5%

26-30

271.5%

409.6%

249.9%

308.6%

321.0%

213.6%

245.1%

271.5%

371.4%

299.0%

236.4%

304.0%

345.4%

247.5%

31-35

206.3%

362.9%

189.9%

234.5%

243.9%

193.3%

186.3%

206.3%

260.1%

256.6%

179.7%

231.0%

243.9%

185.8%

36-40

136.9%

257.8%

126.0%

155.6%

185.7%

107.7%

123.6%

136.9%

172.6%

194.6%

119.2%

152.7%

142.6%

123.4%

41-45

159.2%

240.1%

146.5%

180.9%

239.5%

125.2%

143.7%

159.2%

200.7%

261.2%

138.6%

178.2%

143.5%

143.4%

46-50

163.2%

236.6%

150.2%

185.5%

193.0%

128.4%

147.3%

163.2%

206.9%

237.3%

135.0%

178.1%

172.3%

168.3%

51-55

176.9%

241.0%

162.8%

223.6%

226.1%

139.2%

159.7%

179.4%

223.0%

251.4%

140.9%

210.2%

161.2%

159.4%

56-60

151.7%

227.7%

139.7%

188.4%

184.3%

119.4%

128.1%

112.6%

191.6%

237.7%

132.1%

152.3%

164.2%

146.3%

61-65

132.4%

207.2%

120.0%

138.2%

156.5%

101.3%

116.0%

135.3%

186.8%

198.3%

115.3%

148.2%

136.9%

119.2%

66-70

143.9%

245.9%

130.4%

165.3%

170.1%

113.2%

129.9%

143.9%

171.4%

230.4%

135.2%

161.1%

161.6%

151.5%

71-75

151.9%

239.6%

139.8%

161.0%

163.7%

119.5%

151.9%

151.9%

150.4%

222.9%

132.3%

170.1%

162.0%

136.8%

76-80

144.6%

226.8%

133.1%

164.3%

155.2%

113.8%

129.2%

144.6%

138.5%

211.0%

142.1%

187.4%

157.4%

113.3%

81-85

143.0%

215.8%

156.8%

159.8%

169.1%

154.6%

149.9%

143.0%

137.7%

197.4%

124.6%

160.2%

151.0%

98.1%

86-90

123.1%

157.0%

113.3%

139.9%

145.5%

96.8%

111.1%

123.1%

155.1%

174.9%

107.2%

137.8%

129.9%

88.9%

91-95

108.5%

163.7%

99.8%

123.3%

128.3%

85.3%

97.9%

108.5%

136.8%

165.0%

94.5%

121.5%

114.6%

97.7%

96-100

131.6%

198.6%

121.1%

149.6%

155.6%

103.6%

118.8%

131.6%

165.9%

187.1%

114.6%

147.4%

139.0%

118.6%

101-105

146.0%

220.3%

134.4%

166.0%

172.7%

114.9%

131.8%

146.0%

184.1%

207.6%

127.2%

163.5%

154.2%

131.5%

106-110

146.0%

220.3%

134.4%

166.0%

172.7%

114.9%

131.8%

146.0%

184.1%

207.6%

127.2%

163.5%

154.2%

131.5%

111-115

146.0%

220.3%

134.4%

166.0%

172.7%

114.9%

131.8%

146.0%

184.1%

207.6%

127.2%

163.5%

154.2%

131.5%

Sex

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

F

132.4%

207.2%

120.0%

138.2%

156.5%

101.3%

116.0%

135.3%

186.8%

198.3%

115.3%

148.2%

136.9%

119.2%

M

161.7%

266.7%

146.2%

161.0%

185.9%

123.7%

145.0%

178.7%

229.1%

242.2%

126.2%

175.4%

159.5%

145.6%

Dur_Grp

WLT

NLT

ULT

1-1

149.8%

138.2%

202.2%

2-2

134.7%

211.3%

181.8%

3-3

155.6%

128.1%

185.9%

4-5

116.0%

107.0%

143.9%

6-10

117.0%

114.3%

134.1%

11-15

128.9%

118.2%

127.0%

16-20

115.7%

106.8%

111.9%

21-30

99.0%

91.3%

109.2%

31-100

93.0%

85.8%

109.9%

Face_Amount_Band

F

M

<10K

114.7%

143.4%

10K - <25K

116.0%

145.0%

25K - <50K

104.3%

136.1%

50K - <100K

84.7%

108.1%

AA_Grp

1-1

2-2

3-3

4-5

6-10

11-15

16-20

21-30

31-100

0-17

165.6%

162.8%

172.0%

131.8%

133.2%

131.8%

116.6%

109.4%

109.7%

18-25

165.6%

142.4%

129.4%

131.8%

136.0%

128.7%

116.6%

109.4%

109.7%

26-30

232.4%

228.5%

244.8%

245.1%

215.9%

185.0%

163.7%

153.6%

153.9%

31-35

233.9%

218.1%

244.3%

186.3%

225.4%

186.2%

164.8%

154.6%

154.9%

36-40

178.9%

188.0%

159.1%

123.6%

162.5%

155.5%

108.7%

118.3%

118.5%

41-45

180.5%

171.8%

189.0%

143.7%

140.2%

148.6%

153.6%

149.3%

119.5%

46-50

185.0%

192.8%

206.7%

147.3%

175.6%

145.9%

130.3%

129.7%

164.2%

51-55

195.9%

189.8%

203.5%

159.7%

157.6%

156.0%

154.6%

117.2%

132.6%

56-60

160.9%

149.2%

165.6%

128.1%

140.8%

132.5%

125.1%

102.5%

96.5%

61-65

149.8%

134.7%

155.6%

116.0%

117.0%

128.9%

115.7%

99.0%

93.0%

66-70

166.8%

164.0%

170.6%

129.9%

120.1%

132.8%

123.4%

113.0%

110.5%

71-75

187.3%

211.2%

191.9%

151.9%

137.0%

136.1%

117.7%

123.8%

119.1%

76-80

162.2%

215.0%

168.5%

129.2%

114.1%

124.3%

101.4%

104.1%

104.1%

81-85

188.2%

280.7%

241.6%

149.9%

127.5%

135.2%

117.2%

124.4%

121.8%

86-90

139.5%

137.2%

144.9%

111.1%

112.2%

111.1%

93.6%

92.2%

92.5%

91-95

123.0%

121.0%

127.8%

97.9%

98.9%

97.9%

86.6%

80.3%

81.5%

96-100

149.2%

146.8%

155.0%

118.8%

120.0%

118.8%

105.1%

98.6%

98.8%

101-105

165.6%

162.8%

172.0%

131.8%

133.2%

131.8%

116.6%

109.4%

109.7%

106-110

165.6%

162.8%

172.0%

131.8%

133.2%

131.8%

116.6%

109.4%

109.7%

111-115

165.6%

162.8%

172.0%

131.8%

133.2%

131.8%

116.6%

109.4%

109.7%

Dur_Grp

F

M

1-1

149.8%

180.7%

2-2

134.7%

162.3%

3-3

155.6%

187.7%

4-5

116.0%

145.0%

6-10

117.0%

146.7%

11-15

128.9%

155.4%

16-20

115.7%

140.5%

21-30

99.0%

119.1%

31-100

93.0%

115.0%

AA_Grp

WLT

NLT

ULT

0-17

131.8%

108.1%

144.4%

18-25

131.8%

108.1%

167.3%

26-30

245.1%

201.1%

280.4%

31-35

186.3%

152.8%

165.4%

36-40

123.6%

101.4%

126.3%

41-45

143.7%

105.3%

135.8%

46-50

147.3%

114.2%

166.2%

51-55

159.7%

139.0%

174.1%

56-60

128.1%

105.1%

140.3%

61-65

116.0%

107.0%

143.9%

66-70

129.9%

106.5%

151.6%

71-75

151.9%

138.2%

211.3%

76-80

129.2%

111.0%

179.7%

81-85

149.9%

122.9%

215.9%

86-90

111.1%

91.1%

160.5%

91-95

97.9%

80.3%

101.3%

96-100

118.8%

97.5%

117.5%

101-105

131.8%

108.1%

144.4%

106-110

131.8%

108.1%

144.4%

111-115

131.8%

108.1%

144.4%

SOA_Post_Lvl_Ind

F

M

WLT

116.0%

145.0%

NLT

107.0%

121.2%

ULT

143.9%

184.2%

AA_Grp

F

M

0-17

131.8%

157.3%

18-25

131.8%

176.1%

26-30

245.1%

309.1%

31-35

186.3%

293.8%

36-40

123.6%

202.5%

41-45

143.7%

181.0%

46-50

147.3%

183.6%

51-55

159.7%

215.5%

56-60

128.1%

156.9%

61-65

116.0%

145.0%

66-70

129.9%

155.0%

71-75

151.9%

164.5%

76-80

129.2%

140.2%

81-85

149.9%

167.8%

86-90

111.1%

123.8%

91-95

97.9%

113.9%

96-100

118.8%

139.4%

101-105

131.8%

157.3%

106-110

131.8%

157.3%

111-115

131.8%

157.3%

7.1.6.2 Term Model 100K+, Excluding PLT

As with the under 100K model, the elastic net model is fitted. Note the use of the sparse option. This enables efficient matrix algebra when using the elastic net routines.

Code
dat.term.noplt[
                  !(Face_Amount_Band %in% c("<10K",
    "10K - <25K",
    "25K - <50K",
    "50K - <100K"))
  ] %>%
  mutate(Face_Amount_Band=fct_drop(Face_Amount_Band)) %>%
  prepELData(
    formula=glmnetFormula,
    data=.,
    predictors = pred.cols,
    response = "AE_Amount",
    weights = "ExpDth_Amt_VBT2015",
    useSparse = T,
    dropunused = F
  ) -> dat.term.noplt.100kp


set.seed(cvfit.seed)
if(bUseCache & file.exists(
  'term.model.el.noplt.100kp.rds'
) & !bInvalidateCaches)
{
  cvfit.term.noplt.100kp <- readRDS('term.model.el.noplt.100kp.rds')
} else
{
  cvfit.term.noplt.100kp <- fitCVGLMNet(
    dat.term.noplt.100kp,
    nfolds = nFolds
  )
  
  if(bUseCache)
    saveRDS(cvfit.term.noplt.100kp, 'term.model.el.noplt.100kp.rds')
}
7.1.6.2.1 Usual plots for Elastic Net Models

When presenting elastic net models, the cross validation plot for \(\lambda\) and the coefficient shrinkage plots are provided.

At the minimum \(\lambda\) of 0.00128, the model has 124 parameters.

Code
plot(cvfit.term.noplt.100kp)

Code
plot(cvfit.term.noplt.100kp$glmnet.fit,xvar="lambda")

7.1.6.2.2 Factor Table

Their exponentiated coefficients are as follows:

Code
reformatCoefs(cvfit.term.noplt.100kp, pred.cols)  %>%
  filter(Coef != 0) %>%
  select(Feature1Name,
         Feature1Level,
         Feature2Name,
         Feature2Level,
         Coef) %>%
  mutate(Coef=exp(Coef)) %>%
  flextable() %>%
  set_formatter(
    Coef=function(x) paste0(sprintf("%.01f", 100*x),"%")
  ) %>%
  theme_vanilla()

Feature1Name

Feature1Level

Feature2Name

Feature2Level

Coef

(Intercept)

(Intercept)

88.5%

Sex

F

94.4%

Sex

M

102.7%

Face_Amount_Band

250K - <500K

94.3%

Face_Amount_Band

500K - <1M

89.7%

Face_Amount_Band

1M - <2.5M

88.5%

Face_Amount_Band

2.5M - <5M

91.6%

Face_Amount_Band

10M+

77.7%

SOA_Post_Lvl_Ind

NLT

91.2%

UW

NS/2/2

148.3%

UW

NS/1/3

83.3%

UW

NS/3/3

123.8%

UW

NS/1/4

87.4%

UW

NS/3/4

112.0%

UW

NS/4/4

128.6%

UW

S/2/2

121.7%

UW

S/U/U

114.3%

AA_Grp

26-30

100.7%

AA_Grp

56-60

96.2%

AA_Grp

76-80

102.8%

Dur_Grp

2-2

100.5%

Dur_Grp

6-10

99.9%

Dur_Grp

11-15

97.7%

Dur_Grp

16-20

94.5%

Dur_Grp

21-30

111.2%

Dur_Grp

31-100

106.8%

Face_Amount_Band

500K - <1M

UW

NS/2/2

98.3%

Face_Amount_Band

2.5M - <5M

UW

NS/1/3

89.8%

Face_Amount_Band

5M - <10M

UW

NS/1/3

100.8%

Face_Amount_Band

1M - <2.5M

UW

NS/2/3

100.0%

Face_Amount_Band

2.5M - <5M

UW

NS/2/3

102.2%

Face_Amount_Band

1M - <2.5M

UW

NS/3/3

108.0%

Face_Amount_Band

1M - <2.5M

UW

NS/1/4

96.8%

Face_Amount_Band

10M+

UW

NS/1/4

130.2%

Face_Amount_Band

500K - <1M

UW

NS/2/4

100.4%

Face_Amount_Band

1M - <2.5M

UW

NS/2/4

101.8%

Face_Amount_Band

2.5M - <5M

UW

NS/2/4

96.1%

Face_Amount_Band

10M+

UW

NS/2/4

86.1%

Face_Amount_Band

500K - <1M

UW

NS/3/4

104.8%

Face_Amount_Band

1M - <2.5M

UW

NS/3/4

102.5%

Face_Amount_Band

1M - <2.5M

UW

NS/4/4

103.8%

Face_Amount_Band

2.5M - <5M

UW

NS/4/4

105.8%

Face_Amount_Band

10M+

Dur_Grp

4-5

76.3%

Face_Amount_Band

1M - <2.5M

Dur_Grp

6-10

99.3%

Face_Amount_Band

2.5M - <5M

Dur_Grp

6-10

93.5%

Face_Amount_Band

5M - <10M

Dur_Grp

6-10

98.3%

Face_Amount_Band

10M+

Dur_Grp

6-10

120.4%

Face_Amount_Band

500K - <1M

Dur_Grp

11-15

99.4%

Face_Amount_Band

5M - <10M

Dur_Grp

11-15

103.7%

Face_Amount_Band

5M - <10M

Dur_Grp

16-20

92.1%

Face_Amount_Band

1M - <2.5M

AA_Grp

36-40

96.7%

Face_Amount_Band

10M+

AA_Grp

36-40

120.1%

Face_Amount_Band

1M - <2.5M

AA_Grp

41-45

98.3%

Face_Amount_Band

1M - <2.5M

AA_Grp

46-50

99.7%

Face_Amount_Band

250K - <500K

AA_Grp

51-55

100.3%

Face_Amount_Band

10M+

AA_Grp

51-55

94.3%

Face_Amount_Band

250K - <500K

AA_Grp

56-60

97.8%

Face_Amount_Band

500K - <1M

AA_Grp

56-60

98.0%

Face_Amount_Band

10M+

AA_Grp

56-60

101.8%

Face_Amount_Band

250K - <500K

AA_Grp

61-65

99.5%

Face_Amount_Band

1M - <2.5M

AA_Grp

61-65

101.2%

Face_Amount_Band

250K - <500K

AA_Grp

66-70

99.8%

Face_Amount_Band

500K - <1M

AA_Grp

66-70

99.0%

Face_Amount_Band

1M - <2.5M

AA_Grp

66-70

107.4%

Face_Amount_Band

10M+

AA_Grp

71-75

98.5%

UW

NS/3/4

Dur_Grp

2-2

100.6%

UW

NS/1/4

Dur_Grp

4-5

100.8%

UW

NS/U/U

Dur_Grp

4-5

105.8%

UW

NS/3/3

Dur_Grp

6-10

101.8%

UW

S/1/2

Dur_Grp

6-10

97.9%

UW

NS/2/3

Dur_Grp

11-15

100.0%

UW

NS/1/4

Dur_Grp

11-15

95.8%

UW

NS/3/4

Dur_Grp

11-15

102.3%

UW

NS/U/U

Dur_Grp

11-15

96.2%

UW

NS/2/2

Dur_Grp

16-20

103.8%

UW

NS/3/3

AA_Grp

36-40

104.3%

UW

NS/2/4

AA_Grp

36-40

101.5%

UW

NS/1/4

AA_Grp

46-50

102.2%

UW

NS/4/4

AA_Grp

46-50

109.1%

UW

NS/1/4

AA_Grp

51-55

93.9%

UW

NS/U/U

AA_Grp

51-55

96.4%

UW

NS/1/4

AA_Grp

56-60

101.5%

UW

NS/2/4

AA_Grp

56-60

99.9%

UW

NS/1/4

AA_Grp

61-65

97.5%

UW

NS/2/4

AA_Grp

61-65

100.2%

UW

NS/2/4

AA_Grp

66-70

99.5%

UW

NS/4/4

AA_Grp

66-70

99.0%

Sex

M

UW

NS/2/2

104.8%

Sex

M

UW

NS/3/3

107.2%

Sex

M

UW

NS/1/4

98.0%

Sex

M

UW

NS/3/4

104.2%

Sex

M

UW

NS/4/4

113.1%

SOA_Post_Lvl_Ind

ULT

Dur_Grp

6-10

96.2%

SOA_Post_Lvl_Ind

ULT

Dur_Grp

16-20

110.8%

SOA_Post_Lvl_Ind

ULT

Dur_Grp

21-30

116.7%

SOA_Post_Lvl_Ind

ULT

Dur_Grp

31-100

107.1%

Sex

M

Face_Amount_Band

250K - <500K

97.8%

Sex

M

Face_Amount_Band

500K - <1M

97.2%

Sex

M

Face_Amount_Band

1M - <2.5M

94.3%

Sex

M

Face_Amount_Band

5M - <10M

98.5%

AA_Grp

46-50

Dur_Grp

4-5

107.7%

AA_Grp

51-55

Dur_Grp

4-5

97.8%

AA_Grp

61-65

Dur_Grp

4-5

102.7%

AA_Grp

36-40

Dur_Grp

6-10

95.4%

AA_Grp

51-55

Dur_Grp

6-10

100.3%

AA_Grp

56-60

Dur_Grp

6-10

106.3%

AA_Grp

61-65

Dur_Grp

6-10

91.1%

AA_Grp

66-70

Dur_Grp

6-10

99.0%

AA_Grp

56-60

Dur_Grp

11-15

99.1%

Sex

M

Dur_Grp

4-5

102.3%

Sex

M

Dur_Grp

16-20

98.6%

SOA_Post_Lvl_Ind

ULT

AA_Grp

36-40

97.0%

SOA_Post_Lvl_Ind

ULT

AA_Grp

46-50

95.7%

SOA_Post_Lvl_Ind

ULT

AA_Grp

61-65

101.6%

SOA_Post_Lvl_Ind

ULT

AA_Grp

66-70

115.2%

SOA_Post_Lvl_Ind

ULT

AA_Grp

71-75

108.3%

Sex

M

SOA_Post_Lvl_Ind

NLT

96.5%

Sex

M

AA_Grp

31-35

106.3%

Sex

M

AA_Grp

36-40

110.4%

Sex

M

AA_Grp

41-45

99.6%

Sex

M

AA_Grp

51-55

101.2%

Sex

M

AA_Grp

61-65

96.5%

Sex

M

AA_Grp

66-70

93.5%

Sex

M

AA_Grp

76-80

103.3%

Plots and tables of effects are much more digestible.

In the plots and tables that follow, we fix all of the other variables at their middle values when extracting final factors.

We generate some supporting tables: a factor grid for attaching to the experience, and a list of interactions present in the model

Code
dat.term.noplt[!(Face_Amount_Band %in% c("<10K",
    "10K - <25K",
    "25K - <50K",
    "50K - <100K")),..pred.cols] %>%
  mutate(Face_Amount_Band=fct_drop(Face_Amount_Band)) %>%
  lapply(levels) %>%
  expand.grid() %>%
  setDT() -> 
  dat.term.noplt.100kp.grid 

dat.term.noplt.100kp.grid %>%
  model.Matrix(
    object=glmnetFormula,
    data=.,
    sparse=T
  ) %>%
  predict(
    cvfit.term.noplt.100kp,
    newx=.,
    s="lambda.min"
  ) %>% 
  as.vector() ->
  newCoef

dat.term.noplt.100kp.grid %>%
  add_column(
    Factor=exp(newCoef)
  )  %>% 
  setDT() -> 
  dat.term.noplt.100kp.grid 

write.xlsx(dat.term.noplt.100kp.grid,
           file="dat.term.noplt.100kp.grid.xlsx")

reformatCoefs(cvfit.term.noplt.100kp, pred.cols) %>%
  filter(Coef != 0 & !is.na(Feature2Name)) %>% 
  select(Feature1Name,Feature2Name) %>% 
  distinct() %>%
  as.list() %>%
  purrr::list_transpose() ->
  term.no.plt.100kp.int.list
7.1.6.2.3 Plots of Terms

Below are plots of the 2-way interaction terms, with external factors fixed at their middle values.

The interactions tend to be subtle. To that end, some larger ones are noted.

  1. Duration x Post-Level Indicator: Later duration unknown level term stands out.
  2. Attained Age x Post-Level Indicator: Middle attained ages for unknown level term.
  3. Attained Age x Sex: Some larger deviations for males between ages 26 and 80.
7.1.6.2.4 Tables of Terms

Below are tables of the 2-way interaction terms, with external factors fixed at their middle values.

Face_Amount_Band

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

100K - <250K

85.8%

127.2%

71.5%

85.8%

106.2%

73.6%

86.0%

96.1%

110.3%

90.8%

85.8%

104.4%

98.1%

85.8%

250K - <500K

80.6%

119.5%

67.1%

80.6%

99.7%

69.1%

80.8%

90.2%

103.6%

85.2%

80.6%

98.0%

92.1%

80.6%

500K - <1M

77.0%

112.3%

64.2%

77.0%

95.3%

66.1%

77.5%

90.4%

99.0%

81.5%

77.0%

93.7%

88.0%

77.0%

1M - <2.5M

76.8%

113.9%

64.0%

76.8%

102.7%

63.8%

78.4%

88.2%

102.6%

81.3%

76.8%

93.5%

87.8%

76.8%

2.5M - <5M

78.6%

116.5%

58.8%

80.3%

97.3%

67.4%

75.7%

88.0%

106.9%

83.1%

78.6%

95.6%

89.9%

78.6%

5M - <10M

85.8%

127.2%

72.0%

85.8%

106.2%

73.6%

86.0%

96.1%

110.3%

90.8%

85.8%

104.4%

98.1%

85.8%

10M+

50.9%

75.4%

42.4%

50.9%

63.0%

56.8%

43.9%

57.0%

65.4%

53.8%

50.9%

61.9%

58.1%

50.9%

Dur_Grp

100K - <250K

250K - <500K

500K - <1M

1M - <2.5M

2.5M - <5M

5M - <10M

10M+

1-1

83.8%

78.7%

75.5%

76.3%

73.8%

83.8%

56.0%

2-2

84.2%

79.0%

75.8%

76.7%

74.1%

84.2%

56.3%

3-3

83.8%

78.7%

75.5%

76.3%

73.8%

83.8%

56.0%

4-5

86.0%

80.8%

77.5%

78.4%

75.7%

86.0%

43.9%

6-10

76.2%

71.6%

68.7%

69.0%

62.7%

74.9%

61.4%

11-15

81.9%

76.9%

73.3%

74.6%

72.1%

84.9%

54.8%

16-20

79.2%

74.3%

71.3%

72.1%

69.7%

72.9%

53.0%

21-30

93.2%

87.5%

83.9%

84.9%

82.1%

93.2%

62.3%

31-100

89.5%

84.0%

80.6%

81.5%

78.8%

89.5%

59.9%

AA_Grp

100K - <250K

250K - <500K

500K - <1M

1M - <2.5M

2.5M - <5M

5M - <10M

10M+

0-17

83.6%

78.8%

75.3%

75.3%

73.6%

83.6%

42.7%

18-25

83.6%

78.8%

75.3%

75.3%

73.6%

83.6%

42.7%

26-30

84.1%

79.4%

75.8%

75.8%

74.1%

84.1%

43.0%

31-35

83.6%

78.8%

75.3%

75.3%

73.6%

83.6%

42.7%

36-40

84.8%

80.0%

76.4%

73.9%

74.7%

84.8%

52.0%

41-45

83.6%

78.8%

75.3%

74.0%

73.6%

83.6%

42.7%

46-50

90.0%

84.9%

81.1%

80.9%

79.3%

90.0%

46.0%

51-55

81.8%

77.3%

73.7%

73.7%

72.0%

81.8%

39.4%

56-60

80.3%

74.1%

70.9%

72.4%

70.8%

80.3%

41.8%

61-65

86.0%

80.8%

77.5%

78.4%

75.7%

86.0%

43.9%

66-70

83.2%

78.3%

74.2%

80.4%

73.2%

83.2%

42.5%

71-75

83.6%

78.8%

75.3%

75.3%

73.6%

83.6%

42.0%

76-80

85.9%

81.1%

77.4%

77.4%

75.7%

85.9%

43.9%

81-85

83.6%

78.8%

75.3%

75.3%

73.6%

83.6%

42.7%

86-90

83.6%

78.8%

75.3%

75.3%

73.6%

83.6%

42.7%

91-95

83.6%

78.8%

75.3%

75.3%

73.6%

83.6%

42.7%

96-100

83.6%

78.8%

75.3%

75.3%

73.6%

83.6%

42.7%

101-105

83.6%

78.8%

75.3%

75.3%

73.6%

83.6%

42.7%

106-110

83.6%

78.8%

75.3%

75.3%

73.6%

83.6%

42.7%

111-115

83.6%

78.8%

75.3%

75.3%

73.6%

83.6%

42.7%

Dur_Grp

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

1-1

75.0%

109.4%

62.5%

75.0%

92.8%

63.9%

75.5%

88.1%

96.5%

75.0%

75.0%

91.3%

85.8%

75.0%

2-2

75.4%

109.9%

62.8%

75.4%

93.3%

64.2%

75.8%

89.0%

96.9%

75.4%

75.4%

91.7%

86.2%

75.4%

3-3

75.0%

109.4%

62.5%

75.0%

92.8%

63.9%

75.5%

88.1%

96.5%

75.0%

75.0%

91.3%

85.8%

75.0%

4-5

77.0%

112.3%

64.2%

77.0%

95.3%

66.1%

77.5%

90.4%

99.0%

81.5%

77.0%

93.7%

88.0%

77.0%

6-10

68.2%

99.5%

56.9%

68.2%

86.0%

58.1%

68.7%

80.1%

87.8%

68.2%

66.8%

83.0%

78.0%

68.2%

11-15

72.9%

106.3%

60.7%

72.9%

90.2%

59.4%

73.3%

87.5%

93.7%

70.1%

72.9%

88.7%

83.3%

72.9%

16-20

70.9%

107.3%

59.1%

70.9%

87.7%

60.3%

71.3%

83.2%

91.2%

70.9%

70.9%

86.2%

81.0%

70.9%

21-30

83.4%

121.7%

69.5%

83.4%

103.3%

71.0%

83.9%

98.0%

107.3%

83.4%

83.4%

101.5%

95.4%

83.4%

31-100

80.1%

116.8%

66.8%

80.1%

99.2%

68.2%

80.6%

94.1%

103.1%

80.1%

80.1%

97.5%

91.6%

80.1%

AA_Grp

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

0-17

75.0%

109.4%

62.5%

75.0%

92.8%

66.0%

75.3%

88.1%

96.5%

79.4%

75.0%

91.3%

85.8%

75.0%

18-25

75.0%

109.4%

62.5%

75.0%

92.8%

66.0%

75.3%

88.1%

96.5%

79.4%

75.0%

91.3%

85.8%

75.0%

26-30

75.5%

110.1%

62.9%

75.5%

93.5%

66.5%

75.8%

88.7%

97.1%

79.9%

75.5%

91.9%

86.3%

75.5%

31-35

75.0%

109.4%

62.5%

75.0%

92.8%

66.0%

75.3%

88.1%

96.5%

79.4%

75.0%

91.3%

85.8%

75.0%

36-40

75.0%

109.4%

62.5%

75.0%

96.9%

66.0%

76.4%

88.1%

96.5%

79.4%

75.0%

91.3%

85.8%

75.0%

41-45

75.0%

109.4%

62.5%

75.0%

92.8%

66.0%

75.3%

88.1%

96.5%

79.4%

75.0%

91.3%

85.8%

75.0%

46-50

80.8%

117.8%

67.3%

80.8%

100.0%

72.7%

81.1%

94.9%

113.3%

85.5%

80.8%

98.3%

92.4%

80.8%

51-55

73.4%

107.0%

61.2%

73.4%

90.8%

60.7%

73.7%

86.2%

94.4%

74.9%

73.4%

89.3%

83.9%

73.4%

56-60

70.7%

103.1%

58.9%

70.7%

87.5%

63.1%

70.9%

83.0%

90.9%

74.8%

70.7%

86.0%

80.8%

70.7%

61-65

77.0%

112.3%

64.2%

77.0%

95.3%

66.1%

77.5%

90.4%

99.0%

81.5%

77.0%

93.7%

88.0%

77.0%

66-70

74.3%

108.3%

61.9%

74.3%

91.9%

65.4%

74.2%

87.2%

94.5%

78.6%

74.3%

90.4%

84.9%

74.3%

71-75

75.0%

109.4%

62.5%

75.0%

92.8%

66.0%

75.3%

88.1%

96.5%

79.4%

75.0%

91.3%

85.8%

75.0%

76-80

77.1%

112.5%

64.3%

77.1%

95.5%

67.9%

77.4%

90.6%

99.2%

81.6%

77.1%

93.9%

88.2%

77.1%

81-85

75.0%

109.4%

62.5%

75.0%

92.8%

66.0%

75.3%

88.1%

96.5%

79.4%

75.0%

91.3%

85.8%

75.0%

86-90

75.0%

109.4%

62.5%

75.0%

92.8%

66.0%

75.3%

88.1%

96.5%

79.4%

75.0%

91.3%

85.8%

75.0%

91-95

75.0%

109.4%

62.5%

75.0%

92.8%

66.0%

75.3%

88.1%

96.5%

79.4%

75.0%

91.3%

85.8%

75.0%

96-100

75.0%

109.4%

62.5%

75.0%

92.8%

66.0%

75.3%

88.1%

96.5%

79.4%

75.0%

91.3%

85.8%

75.0%

101-105

75.0%

109.4%

62.5%

75.0%

92.8%

66.0%

75.3%

88.1%

96.5%

79.4%

75.0%

91.3%

85.8%

75.0%

106-110

75.0%

109.4%

62.5%

75.0%

92.8%

66.0%

75.3%

88.1%

96.5%

79.4%

75.0%

91.3%

85.8%

75.0%

111-115

75.0%

109.4%

62.5%

75.0%

92.8%

66.0%

75.3%

88.1%

96.5%

79.4%

75.0%

91.3%

85.8%

75.0%

Sex

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

F

77.0%

112.3%

64.2%

77.0%

95.3%

66.1%

77.5%

90.4%

99.0%

81.5%

77.0%

93.7%

88.0%

77.0%

M

80.4%

122.9%

67.0%

80.4%

106.7%

67.6%

80.9%

98.4%

117.0%

85.0%

80.4%

97.8%

91.9%

80.4%

Dur_Grp

WLT

NLT

ULT

1-1

75.5%

68.8%

76.7%

2-2

75.8%

69.2%

77.1%

3-3

75.5%

68.8%

76.7%

4-5

77.5%

70.7%

78.7%

6-10

68.7%

62.6%

67.1%

11-15

73.3%

66.9%

74.5%

16-20

71.3%

65.1%

80.3%

21-30

83.9%

76.6%

99.6%

31-100

80.6%

73.5%

87.7%

Face_Amount_Band

F

M

100K - <250K

86.0%

92.4%

250K - <500K

80.8%

84.9%

500K - <1M

77.5%

80.9%

1M - <2.5M

78.4%

79.4%

2.5M - <5M

75.7%

81.3%

5M - <10M

86.0%

91.0%

10M+

43.9%

47.2%

AA_Grp

1-1

2-2

3-3

4-5

6-10

11-15

16-20

21-30

31-100

0-17

75.3%

75.7%

75.3%

75.3%

75.2%

73.2%

71.1%

83.7%

80.4%

18-25

75.3%

75.7%

75.3%

75.3%

75.2%

73.2%

71.1%

83.7%

80.4%

26-30

75.8%

76.1%

75.8%

75.8%

75.7%

73.6%

71.6%

84.3%

80.9%

31-35

75.3%

75.7%

75.3%

75.3%

75.2%

73.2%

71.1%

83.7%

80.4%

36-40

76.4%

76.8%

76.4%

76.4%

72.8%

74.3%

72.2%

85.0%

81.6%

41-45

75.3%

75.7%

75.3%

75.3%

75.2%

73.2%

71.1%

83.7%

80.4%

46-50

75.3%

75.7%

75.3%

81.1%

75.2%

73.2%

71.1%

83.7%

80.4%

51-55

75.3%

75.7%

75.3%

73.7%

75.4%

73.2%

71.1%

83.7%

80.4%

56-60

70.9%

71.3%

70.9%

70.9%

75.3%

68.3%

67.0%

78.9%

75.8%

61-65

75.5%

75.8%

75.5%

77.5%

68.7%

73.3%

71.3%

83.9%

80.6%

66-70

74.2%

74.5%

74.2%

74.2%

73.4%

72.1%

70.1%

82.5%

79.2%

71-75

75.3%

75.7%

75.3%

75.3%

75.2%

73.2%

71.1%

83.7%

80.4%

76-80

77.4%

77.8%

77.4%

77.4%

77.3%

75.2%

73.2%

86.1%

82.7%

81-85

75.3%

75.7%

75.3%

75.3%

75.2%

73.2%

71.1%

83.7%

80.4%

86-90

75.3%

75.7%

75.3%

75.3%

75.2%

73.2%

71.1%

83.7%

80.4%

91-95

75.3%

75.7%

75.3%

75.3%

75.2%

73.2%

71.1%

83.7%

80.4%

96-100

75.3%

75.7%

75.3%

75.3%

75.2%

73.2%

71.1%

83.7%

80.4%

101-105

75.3%

75.7%

75.3%

75.3%

75.2%

73.2%

71.1%

83.7%

80.4%

106-110

75.3%

75.7%

75.3%

75.3%

75.2%

73.2%

71.1%

83.7%

80.4%

111-115

75.3%

75.7%

75.3%

75.3%

75.2%

73.2%

71.1%

83.7%

80.4%

Dur_Grp

F

M

1-1

75.5%

77.0%

2-2

75.8%

77.4%

3-3

75.5%

77.0%

4-5

77.5%

80.9%

6-10

68.7%

70.1%

11-15

73.3%

74.8%

16-20

71.3%

71.7%

21-30

83.9%

85.7%

31-100

80.6%

82.3%

AA_Grp

WLT

NLT

ULT

0-17

75.3%

68.7%

75.3%

18-25

75.3%

68.7%

75.3%

26-30

75.8%

69.1%

75.8%

31-35

75.3%

68.7%

75.3%

36-40

76.4%

69.7%

74.1%

41-45

75.3%

68.7%

75.3%

46-50

81.1%

74.0%

77.6%

51-55

73.7%

67.2%

73.7%

56-60

70.9%

64.7%

70.9%

61-65

77.5%

70.7%

78.7%

66-70

74.2%

67.7%

85.4%

71-75

75.3%

68.7%

81.5%

76-80

77.4%

70.6%

77.4%

81-85

75.3%

68.7%

75.3%

86-90

75.3%

68.7%

75.3%

91-95

75.3%

68.7%

75.3%

96-100

75.3%

68.7%

75.3%

101-105

75.3%

68.7%

75.3%

106-110

75.3%

68.7%

75.3%

111-115

75.3%

68.7%

75.3%

SOA_Post_Lvl_Ind

F

M

WLT

77.5%

80.9%

NLT

70.7%

71.2%

ULT

78.7%

82.2%

AA_Grp

F

M

0-17

75.3%

81.5%

18-25

75.3%

81.5%

26-30

75.8%

82.0%

31-35

75.3%

86.6%

36-40

76.4%

91.3%

41-45

75.3%

81.2%

46-50

81.1%

87.8%

51-55

73.7%

80.7%

56-60

70.9%

76.8%

61-65

77.5%

80.9%

66-70

74.2%

75.0%

71-75

75.3%

81.5%

76-80

77.4%

86.6%

81-85

75.3%

81.5%

86-90

75.3%

81.5%

91-95

75.3%

81.5%

96-100

75.3%

81.5%

101-105

75.3%

81.5%

106-110

75.3%

81.5%

111-115

75.3%

81.5%

7.1.6.3 Goodness-of-Fit

Next comes checking goodness-of-fit, on both a univariate and bivariate basis. Tables are provided which show the ratio of the actual to model predicted deaths and show the associated death counts.

Code
### This code attaches the grid of predicted factors
### to the underlying data for further analysis.
dat.term.noplt.lt100k.grid %>%
  rbind(dat.term.noplt.100kp.grid) %>%
  right_join(dat.term.noplt) %>%
  mutate(ExpDth_Amt_GLMNet=ExpDth_Amt_VBT2015*Factor) %>%
  mutate(Factor=NULL) ->
  dat.term.noplt
7.1.6.3.1 Univariate Fit Checks

Sex

Death Count

Actual-to-Model

F

83,020

99.6%

M

164,796

100.1%

Face_Amount_Band

Death Count

Actual-to-Model

<10K

13,432

100.9%

10K - <25K

11,636

100.2%

25K - <50K

16,359

99.9%

50K - <100K

29,178

100.0%

100K - <250K

89,514

102.7%

250K - <500K

47,754

99.6%

500K - <1M

25,324

99.6%

1M - <2.5M

13,090

99.7%

2.5M - <5M

1,011

98.3%

5M - <10M

410

98.7%

10M+

108

96.1%

SOA_Post_Lvl_Ind

Death Count

Actual-to-Model

WLT

184,673

100.0%

NLT

11,320

95.6%

ULT

51,823

100.6%

UW

Death Count

Actual-to-Model

NS/1/2

11,482

92.6%

NS/2/2

41,938

101.5%

NS/1/3

14,303

99.0%

NS/2/3

13,519

100.4%

NS/3/3

21,663

100.9%

NS/1/4

19,687

99.5%

NS/2/4

17,227

100.3%

NS/3/4

11,695

101.1%

NS/4/4

12,580

101.4%

NS/U/U

37,750

100.0%

S/1/2

8,085

97.4%

S/2/2

6,578

104.8%

S/U/U

18,522

103.6%

U/U/U

12,787

112.2%

AA_Grp

Death Count

Actual-to-Model

0-17

165

71.6%

18-25

1,499

97.0%

26-30

2,324

112.7%

31-35

4,025

104.2%

36-40

7,199

98.4%

41-45

13,326

99.6%

46-50

21,638

100.1%

51-55

32,207

100.3%

56-60

38,983

99.6%

61-65

41,361

99.6%

66-70

35,497

99.6%

71-75

24,265

99.2%

76-80

15,044

102.9%

81-85

7,028

107.4%

86-90

2,443

113.7%

91-95

733

109.6%

96-100

78

41.6%

101-105

1

51.8%

106-110

0

0.0%

111-115

0

0.0%

Dur_Grp

Death Count

Actual-to-Model

1-1

5,497

100.6%

2-2

7,141

102.4%

3-3

8,692

100.5%

4-5

19,420

99.6%

6-10

65,368

99.8%

11-15

64,799

99.7%

16-20

39,547

99.3%

21-30

24,005

102.2%

31-100

13,347

115.5%

Unidimensional goodness-of-fit is generally good, with some spots with poor fit. In some cases, it appears to be related to lack of claims, which may be a sign of overpenalization. In other cases, there are enough claims yet fit is genuinely suboptimal, such as UW U/U/U and N/1/2, level term NLT, and durations 31+, among others.

7.1.6.3.2 Bivariate Fit Checks

Face_Amount_Band

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

<10K

4,179

102.3%

9,253

100.1%

10K - <25K

5,989

100.7%

5,647

99.7%

25K - <50K

7,376

99.8%

8,983

100.1%

50K - <100K

12,319

99.9%

16,859

100.0%

100K - <250K

31,763

100.5%

57,751

103.9%

250K - <500K

13,892

100.0%

33,862

99.5%

500K - <1M

5,525

100.0%

19,799

99.5%

1M - <2.5M

1,855

100.1%

11,235

99.6%

2.5M - <5M

74

88.9%

937

99.2%

5M - <10M

39

108.7%

371

97.7%

10M+

9

62.5%

99

100.2%

SOA_Post_Lvl_Ind

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

WLT

62,592

99.7%

122,081

100.1%

NLT

4,110

99.2%

7,210

94.5%

ULT

16,318

99.2%

35,505

101.0%

UW

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

NS/1/2

4,290

93.3%

7,192

92.3%

NS/2/2

20,589

101.2%

21,349

101.6%

NS/1/3

5,068

97.0%

9,235

99.8%

NS/2/3

3,671

100.2%

9,848

100.4%

NS/3/3

5,933

100.6%

15,730

101.0%

NS/1/4

6,928

99.8%

12,759

99.4%

NS/2/4

4,245

99.3%

12,982

100.5%

NS/3/4

2,434

100.4%

9,261

101.2%

NS/4/4

3,434

100.8%

9,146

101.5%

NS/U/U

13,042

100.2%

24,708

99.9%

S/1/2

2,315

98.3%

5,770

97.1%

S/2/2

1,857

108.6%

4,721

103.9%

S/U/U

6,588

105.5%

11,934

102.9%

U/U/U

2,626

103.1%

10,161

115.1%

AA_Grp

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

0-17

67

79.2%

98

68.1%

18-25

392

94.4%

1,107

98.0%

26-30

745

105.1%

1,579

116.1%

31-35

1,494

98.8%

2,531

106.8%

36-40

2,799

88.8%

4,400

102.8%

41-45

5,313

101.7%

8,013

98.7%

46-50

8,274

98.7%

13,364

100.6%

51-55

11,603

99.3%

20,604

100.6%

56-60

13,455

99.8%

25,528

99.5%

61-65

13,390

100.5%

27,971

99.4%

66-70

10,696

101.5%

24,801

99.2%

71-75

7,026

101.5%

17,239

98.8%

76-80

4,117

99.9%

10,927

103.6%

81-85

2,210

118.8%

4,818

103.8%

86-90

1,060

142.9%

1,383

101.2%

91-95

346

119.1%

387

105.9%

96-100

33

41.4%

45

41.7%

101-105

0

0.0%

1

53.6%

106-110

0

0.0%

111-115

0

0.0%

Dur_Grp

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

1-1

1,698

109.9%

3,799

98.6%

2-2

2,346

101.3%

4,795

102.6%

3-3

2,872

102.7%

5,820

99.9%

4-5

6,545

94.9%

12,875

100.9%

6-10

21,943

100.0%

43,425

99.7%

11-15

21,913

98.9%

42,886

100.0%

16-20

14,433

99.9%

25,114

99.2%

21-30

8,425

103.0%

15,580

101.9%

31-100

2,845

110.1%

10,502

117.2%

Face_Amount_Band

SOA_Post_Lvl_Ind: WLT

SOA_Post_Lvl_Ind: NLT

SOA_Post_Lvl_Ind: ULT

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

<10K

531

101.5%

3,752

106.1%

9,149

96.9%

10K - <25K

5,861

100.3%

1,494

99.0%

4,281

100.5%

25K - <50K

9,075

99.9%

1,500

99.4%

5,784

100.2%

50K - <100K

17,862

99.9%

1,398

99.6%

9,918

100.1%

100K - <250K

72,555

102.3%

1,386

94.6%

15,573

105.7%

250K - <500K

42,303

99.8%

1,167

94.9%

4,284

98.8%

500K - <1M

22,935

100.1%

467

93.2%

1,922

96.0%

1M - <2.5M

12,112

100.0%

150

94.4%

828

96.5%

2.5M - <5M

940

97.5%

5

128.0%

66

111.0%

5M - <10M

393

98.2%

1

132.7%

16

110.9%

10M+

106

96.0%

0

0.0%

2

108.1%

UW

Face_Amount_Band: <10K

Face_Amount_Band: 10K - <25K

Face_Amount_Band: 25K - <50K

Face_Amount_Band: 50K - <100K

Face_Amount_Band: 100K - <250K

Face_Amount_Band: 250K - <500K

Face_Amount_Band: 500K - <1M

Face_Amount_Band: 1M - <2

Face_Amount_Band: 2

Face_Amount_Band: 5M - <10M

Face_Amount_Band: 10M+

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

5M

5M - <5M

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

NS/1/2

27

204.2%

44

71.5%

201

97.4%

550

96.8%

6,229

97.9%

2,524

93.5%

1,264

88.3%

565

81.0%

57

101.7%

15

97.6%

6

260.9%

NS/2/2

1,363

105.1%

5,851

99.6%

6,081

100.1%

7,582

100.1%

15,125

111.8%

4,444

98.6%

1,201

92.9%

273

89.5%

14

90.8%

3

55.4%

1

113.8%

NS/1/3

4

59.9%

13

82.0%

62

93.7%

276

99.7%

4,900

100.5%

4,697

99.8%

2,821

101.0%

1,402

97.9%

76

81.2%

45

123.6%

7

71.8%

NS/2/3

5

73.1%

11

104.0%

82

110.5%

616

100.0%

5,154

98.5%

4,152

98.8%

2,129

96.9%

1,212

103.5%

111

116.2%

38

106.2%

9

77.7%

NS/3/3

34

77.9%

118

112.3%

457

98.9%

1,775

100.2%

9,179

99.9%

5,924

100.2%

2,715

100.3%

1,331

103.4%

91

103.5%

33

84.7%

6

117.9%

NS/1/4

1

52.3%

0

0.0%

19

95.8%

412

98.8%

5,595

98.5%

6,113

100.4%

4,467

98.8%

2,716

98.6%

227

94.9%

99

101.2%

38

111.8%

NS/2/4

1

1.6%

3

881.7%

24

91.9%

886

99.5%

6,179

97.6%

4,884

101.6%

3,156

102.9%

1,866

102.2%

140

89.7%

73

106.5%

15

74.9%

NS/3/4

3

128.6%

5

1632.1%

12

82.1%

393

100.9%

4,184

101.4%

3,472

99.1%

2,242

104.3%

1,230

103.5%

101

99.0%

43

95.5%

10

83.5%

NS/4/4

2

235.4%

8

342.1%

287

99.0%

1,003

100.5%

4,964

97.0%

3,412

99.6%

1,732

101.4%

1,016

104.3%

105

118.4%

40

98.0%

11

83.5%

NS/U/U

2,918

105.2%

2,093

100.7%

4,110

99.8%

7,950

100.0%

13,004

106.7%

4,353

99.4%

2,256

98.4%

992

98.4%

55

89.0%

14

68.9%

5

104.4%

S/1/2

4

159.5%

10

414.4%

60

96.3%

776

100.0%

4,752

98.5%

1,574

98.3%

630

101.4%

258

98.9%

17

97.0%

4

64.1%

0

0.0%

S/2/2

6

118.4%

55

113.2%

345

101.8%

975

100.0%

3,556

103.1%

1,046

103.8%

430

122.8%

150

105.6%

13

114.7%

2

42.0%

0

0.0%

S/U/U

1,018

101.7%

2,271

99.3%

3,623

100.1%

4,774

100.1%

5,460

107.2%

1,030

102.6%

263

99.5%

78

107.1%

4

115.2%

1

85.7%

0

0.0%

U/U/U

8,046

95.4%

1,154

100.9%

996

100.3%

1,210

100.4%

1,233

132.3%

129

143.3%

18

84.5%

1

24.8%

0

0.0%

0

0.0%

0

0.0%

AA_Grp

Face_Amount_Band: <10K

Face_Amount_Band: 10K - <25K

Face_Amount_Band: 25K - <50K

Face_Amount_Band: 50K - <100K

Face_Amount_Band: 100K - <250K

Face_Amount_Band: 250K - <500K

Face_Amount_Band: 500K - <1M

Face_Amount_Band: 1M - <2

Face_Amount_Band: 2

Face_Amount_Band: 5M - <10M

Face_Amount_Band: 10M+

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

5M

5M - <5M

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

0-17

27

173.0%

72

111.5%

42

60.3%

20

89.5%

4

79.8%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

18-25

51

141.1%

137

112.5%

136

93.3%

469

100.9%

499

115.7%

160

92.3%

37

78.0%

10

102.5%

0

0.0%

0

0.0%

0

0.0%

26-30

9

70.9%

47

95.2%

177

105.5%

580

100.3%

753

143.8%

511

128.8%

187

99.9%

57

93.6%

3

170.9%

0

0.0%

0

0.0%

31-35

12

122.0%

22

64.9%

224

104.3%

618

100.7%

1,212

140.0%

1,024

105.7%

637

105.1%

257

93.1%

16

111.8%

3

97.6%

0

0.0%

36-40

22

134.2%

47

84.7%

264

101.1%

739

100.6%

1,931

108.5%

2,001

96.5%

1,453

98.5%

688

94.4%

39

80.1%

11

88.2%

4

258.3%

41-45

29

82.1%

102

105.6%

393

102.4%

986

99.9%

3,792

106.7%

3,811

101.9%

2,698

101.9%

1,387

97.2%

90

87.3%

31

104.8%

7

77.6%

46-50

80

141.7%

218

105.7%

647

100.9%

1,696

99.7%

7,105

106.3%

5,872

99.7%

3,778

100.0%

2,015

98.1%

152

96.6%

63

114.6%

12

91.8%

51-55

208

114.7%

528

102.1%

1,388

100.6%

2,994

99.8%

11,777

103.2%

8,141

102.1%

4,551

100.3%

2,335

98.6%

202

108.1%

68

93.6%

15

81.0%

56-60

374

107.5%

1,056

101.6%

2,241

99.6%

4,448

99.9%

15,609

101.7%

8,389

98.0%

4,316

98.0%

2,250

98.8%

195

105.7%

75

92.8%

30

115.0%

61-65

747

110.6%

1,665

99.2%

3,200

99.7%

5,345

99.9%

17,054

101.3%

7,715

97.8%

3,598

99.4%

1,819

102.3%

139

97.2%

62

96.8%

17

87.9%

66-70

919

108.9%

2,134

100.8%

3,340

99.7%

5,010

99.9%

14,657

100.8%

5,654

97.1%

2,327

96.4%

1,296

103.4%

92

90.3%

54

106.1%

14

105.2%

71-75

1,958

95.7%

2,085

100.3%

2,338

100.4%

3,424

100.1%

9,422

98.5%

3,093

99.2%

1,187

98.3%

655

105.2%

65

106.4%

31

100.0%

7

62.3%

76-80

3,733

92.5%

1,779

100.0%

1,300

100.5%

2,011

100.2%

4,412

100.6%

1,097

101.1%

430

108.0%

257

116.8%

12

60.2%

11

87.4%

2

137.8%

81-85

3,308

99.1%

1,079

98.5%

497

101.2%

634

99.6%

1,099

109.4%

240

119.5%

108

132.5%

58

120.2%

4

92.4%

1

38.8%

0

0.0%

86-90

1,413

92.5%

530

101.1%

130

93.5%

164

102.6%

150

101.8%

36

137.0%

14

126.7%

4

90.6%

2

475.5%

91-95

491

119.7%

123

86.9%

34

92.5%

35

88.3%

35

98.5%

10

97.4%

3

155.7%

2

316.4%

96-100

51

88.2%

12

57.8%

7

86.7%

5

76.6%

3

49.8%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

101-105

0

0.0%

1

179.8%

0

0.0%

106-110

0

0.0%

111-115

0

0.0%

Face_Amount_Band

Dur_Grp: 1-1

Dur_Grp: 2-2

Dur_Grp: 3-3

Dur_Grp: 4-5

Dur_Grp: 6-10

Dur_Grp: 11-15

Dur_Grp: 16-20

Dur_Grp: 21-30

Dur_Grp: 31-100

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

<10K

33

272.7%

78

171.2%

70

138.9%

174

128.2%

494

117.7%

545

88.8%

1,777

106.9%

2,367

100.4%

7,894

94.1%

10K - <25K

402

126.0%

520

101.4%

518

103.2%

945

98.1%

2,433

99.3%

1,739

99.6%

1,751

98.8%

2,272

99.1%

1,056

101.0%

25K - <50K

505

103.0%

575

98.1%

576

98.4%

1,132

99.2%

3,095

99.7%

2,771

100.3%

3,186

99.7%

3,413

100.2%

1,106

100.9%

50K - <100K

846

102.4%

894

99.3%

942

99.5%

1,972

100.0%

5,996

99.9%

6,766

99.9%

5,222

99.9%

5,035

99.9%

1,505

100.3%

100K - <250K

1,745

116.4%

2,240

112.1%

2,893

120.2%

6,164

101.3%

22,868

102.7%

26,423

99.9%

16,855

103.3%

8,700

99.3%

1,626

128.1%

250K - <500K

1,028

100.8%

1,425

105.2%

1,829

104.9%

4,398

98.5%

15,613

99.5%

15,074

99.6%

6,734

98.4%

1,518

95.2%

135

154.5%

500K - <1M

554

97.7%

825

103.3%

1,112

104.4%

2,716

99.4%

9,141

99.8%

7,590

98.9%

2,825

98.3%

540

102.4%

21

112.5%

1M - <2.5M

330

91.7%

505

102.2%

666

100.2%

1,651

99.7%

5,092

99.3%

3,563

100.1%

1,138

101.6%

141

102.4%

4

61.0%

2.5M - <5M

34

94.2%

55

107.5%

54

78.8%

177

104.3%

400

96.0%

229

101.2%

48

88.0%

14

316.2%

0

0.0%

5M - <10M

14

90.3%

22

99.1%

26

91.8%

79

109.1%

171

95.0%

87

111.3%

8

43.7%

3

209.0%

0

0.0%

10M+

6

151.7%

2

63.0%

6

68.8%

12

72.0%

65

107.2%

12

80.7%

3

80.1%

2

2853.4%

UW

SOA_Post_Lvl_Ind: WLT

SOA_Post_Lvl_Ind: NLT

SOA_Post_Lvl_Ind: ULT

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

NS/1/2

7,897

92.7%

327

97.4%

3,258

92.1%

NS/2/2

35,517

101.1%

479

77.4%

5,942

106.3%

NS/1/3

12,880

98.6%

1,110

97.6%

313

120.0%

NS/2/3

12,845

100.0%

355

94.3%

319

123.1%

NS/3/3

19,350

100.9%

1,566

94.6%

747

113.6%

NS/1/4

19,621

99.5%

1

25.3%

65

101.1%

NS/2/4

17,181

100.5%

0

0.0%

46

60.8%

NS/3/4

11,607

101.0%

0

0.0%

88

111.1%

NS/4/4

12,474

101.6%

1

24.2%

105

84.9%

NS/U/U

12,376

100.2%

4,649

100.3%

20,725

99.8%

S/1/2

7,348

98.5%

58

88.9%

679

86.7%

S/2/2

5,971

105.1%

78

111.6%

529

100.5%

S/U/U

9,106

105.0%

1,902

98.1%

7,514

101.8%

U/U/U

500

82.8%

794

101.8%

11,493

117.3%

AA_Grp

SOA_Post_Lvl_Ind: WLT

SOA_Post_Lvl_Ind: NLT

SOA_Post_Lvl_Ind: ULT

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

0-17

24

72.3%

0

0.0%

141

71.8%

18-25

840

101.2%

7

134.2%

652

88.2%

26-30

1,533

120.5%

9

85.1%

782

91.9%

31-35

3,156

107.9%

31

131.9%

838

87.0%

36-40

6,076

100.4%

89

89.2%

1,034

85.0%

41-45

11,358

99.9%

253

85.7%

1,715

98.7%

46-50

18,041

100.8%

564

105.0%

3,033

93.1%

51-55

25,867

100.5%

1,085

108.6%

5,255

96.9%

56-60

30,261

99.5%

1,448

89.7%

7,274

101.7%

61-65

31,335

99.2%

1,824

91.9%

8,202

105.0%

66-70

27,024

98.8%

1,681

94.6%

6,792

108.0%

71-75

18,275

98.1%

1,403

91.2%

4,587

121.2%

76-80

8,762

102.0%

1,199

88.9%

5,083

120.1%

81-85

1,889

105.4%

880

117.3%

4,259

115.0%

86-90

228

118.3%

623

90.3%

1,592

112.2%

91-95

4

16.6%

205

88.2%

524

112.6%

96-100

19

99.2%

59

40.6%

101-105

1

51.8%

106-110

0

0.0%

111-115

0

0.0%

Dur_Grp

SOA_Post_Lvl_Ind: WLT

SOA_Post_Lvl_Ind: NLT

SOA_Post_Lvl_Ind: ULT

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

1-1

4,645

102.8%

17

63.1%

835

78.8%

2-2

6,239

102.8%

26

369.2%

876

95.2%

3-3

7,861

101.5%

9

127.0%

822

86.0%

4-5

17,919

100.1%

32

93.9%

1,469

90.4%

6-10

61,422

100.0%

585

105.2%

3,361

93.7%

11-15

58,582

99.8%

2,561

93.7%

3,656

102.5%

16-20

27,730

98.7%

3,853

91.9%

7,964

104.3%

21-30

275

98.5%

3,413

97.0%

20,317

102.4%

31-100

824

97.6%

12,523

115.9%

AA_Grp

UW: NS/U/U

UW: S/U/U

UW: U/U/U

UW: NS/1/2

UW: NS/2/2

UW: NS/1/3

UW: NS/2/3

UW: NS/3/3

UW: NS/1/4

UW: NS/2/4

UW: NS/3/4

UW: NS/4/4

UW: S/1/2

UW: S/2/2

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

0-17

5

58.3%

14

63.2%

146

74.8%

18-25

188

118.0%

156

138.5%

404

100.0%

51

91.9%

264

82.7%

54

84.9%

35

85.5%

114

119.2%

51

70.7%

23

75.6%

16

65.1%

31

86.2%

58

185.9%

54

160.6%

26-30

217

135.0%

284

144.8%

131

158.2%

101

62.6%

711

124.7%

148

130.3%

84

142.0%

155

119.1%

132

94.4%

55

97.3%

48

103.0%

82

113.2%

100

152.3%

76

115.2%

31-35

277

84.2%

391

111.7%

123

122.2%

250

85.0%

986

104.3%

294

108.4%

213

133.8%

360

128.4%

375

101.1%

157

98.3%

137

108.4%

180

117.3%

139

77.3%

143

99.3%

36-40

623

92.9%

501

88.9%

125

201.5%

456

69.3%

1,560

98.2%

548

96.2%

372

89.6%

636

119.8%

744

94.4%

430

120.9%

314

102.3%

381

119.7%

278

84.7%

231

96.2%

41-45

1,364

99.5%

843

99.9%

106

111.6%

906

96.6%

2,435

98.6%

1,112

99.0%

777

101.6%

1,270

108.9%

1,538

99.1%

759

99.7%

669

95.2%

665

99.6%

485

88.4%

397

105.5%

46-50

2,739

93.7%

1,534

99.8%

162

95.5%

1,390

88.2%

3,701

98.9%

1,707

101.6%

1,257

100.4%

1,801

97.2%

2,366

103.2%

1,433

102.8%

1,017

102.8%

1,082

109.9%

799

91.5%

650

107.7%

51-55

4,844

95.4%

2,644

102.1%

300

90.9%

1,914

88.8%

5,303

102.4%

2,256

103.0%

1,715

98.9%

2,520

102.4%

3,124

97.1%

2,118

102.3%

1,615

105.1%

1,532

106.9%

1,355

99.5%

967

111.4%

56-60

6,504

99.3%

3,499

106.8%

707

108.2%

2,001

91.5%

6,339

103.9%

2,217

95.3%

2,090

100.9%

3,054

97.9%

3,340

102.7%

2,705

96.3%

1,943

104.8%

1,919

97.0%

1,543

99.7%

1,122

98.4%

61-65

6,981

100.3%

3,558

102.0%

1,351

116.2%

1,783

107.3%

6,408

101.1%

2,143

94.9%

2,232

97.2%

3,488

95.8%

3,132

96.5%

3,122

103.9%

2,198

102.0%

2,300

101.7%

1,500

94.8%

1,165

100.0%

66-70

5,880

101.8%

2,728

106.4%

1,341

121.3%

1,252

104.6%

5,538

100.9%

1,756

96.2%

2,064

100.7%

3,398

103.5%

2,518

99.3%

2,935

95.2%

1,934

96.4%

2,066

92.1%

1,101

107.7%

986

112.9%

71-75

3,521

111.2%

1,377

114.2%

1,606

117.7%

750

88.1%

3,924

100.5%

1,264

99.5%

1,612

103.5%

2,558

97.4%

1,600

98.1%

2,209

94.0%

1,263

92.9%

1,521

97.3%

532

110.0%

528

104.5%

76-80

2,241

122.2%

586

104.7%

2,888

111.5%

376

145.3%

2,747

108.6%

633

102.4%

849

94.7%

1,570

90.4%

638

108.3%

1,016

99.9%

474

99.3%

651

91.2%

176

112.2%

199

102.1%

81-85

1,337

113.7%

289

75.8%

2,263

107.4%

152

90.6%

1,366

102.1%

157

119.5%

195

129.3%

623

107.6%

120

89.2%

243

118.3%

63

79.1%

154

94.4%

16

98.3%

50

84.7%

86-90

787

91.5%

96

78.1%

749

93.2%

79

144.3%

526

107.8%

11

131.0%

23

209.5%

110

94.2%

9

124.3%

22

198.6%

4

67.4%

15

95.2%

3

569.0%

9

97.8%

91-95

236

122.4%

21

84.2%

326

98.0%

20

179.9%

120

82.5%

3

46.1%

0

0.0%

5

15.9%

0

0.0%

1

111.6%

0

0.0%

1

49.0%

96-100

6

21.1%

1

10.1%

58

50.6%

1

118.7%

10

70.5%

1

198.1%

1

1.8%

0

0.0%

0

0.0%

101-105

1

51.8%

106-110

0

0.0%

111-115

0

0.0%

UW

Dur_Grp: 1-1

Dur_Grp: 2-2

Dur_Grp: 3-3

Dur_Grp: 4-5

Dur_Grp: 6-10

Dur_Grp: 11-15

Dur_Grp: 16-20

Dur_Grp: 21-30

Dur_Grp: 31-100

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

NS/1/2

115

63.2%

182

88.5%

248

78.7%

581

77.7%

2,543

90.5%

2,878

93.5%

3,165

98.0%

1,560

120.9%

210

137.4%

NS/2/2

1,379

98.4%

1,846

92.8%

2,039

108.9%

4,047

105.0%

11,815

97.9%

10,482

101.4%

8,703

105.6%

1,519

101.3%

108

93.4%

NS/1/3

182

84.2%

246

110.7%

341

106.0%

856

92.2%

3,619

99.7%

5,756

99.1%

3,270

98.5%

30

304.6%

3

114.1%

NS/2/3

177

82.5%

202

87.6%

325

103.8%

779

99.7%

3,923

102.1%

5,492

102.5%

2,590

93.9%

31

87.4%

0

0.0%

NS/3/3

470

124.5%

646

106.3%

850

104.2%

1,931

97.7%

7,612

101.9%

7,498

100.0%

2,487

94.3%

163

80.2%

6

58.2%

NS/1/4

403

115.7%

614

93.8%

894

97.3%

2,206

103.4%

7,317

99.0%

6,572

98.2%

1,681

97.9%

0

0.0%

NS/2/4

282

110.5%

511

103.8%

714

100.3%

1,727

100.4%

6,770

100.2%

6,059

100.0%

1,164

95.9%

0

0.0%

NS/3/4

288

79.9%

490

120.5%

642

93.1%

1,541

97.8%

4,781

101.0%

3,417

104.3%

536

103.6%

0

0.0%

NS/4/4

437

106.4%

605

106.9%

809

106.4%

1,777

100.9%

5,206

102.4%

3,046

98.3%

676

85.7%

14

59.0%

10

104.3%

NS/U/U

528

83.2%

636

96.5%

682

116.2%

1,462

121.3%

3,588

101.0%

5,667

97.0%

8,985

99.6%

14,258

98.6%

1,944

117.4%

S/1/2

167

82.7%

233

96.0%

291

93.9%

739

96.6%

2,512

92.6%

2,629

101.7%

1,242

108.2%

254

103.7%

18

98.7%

S/2/2

231

130.5%

244

113.5%

318

102.6%

689

93.8%

2,272

102.3%

1,883

107.5%

720

111.5%

187

120.1%

34

137.3%

S/U/U

459

117.9%

449

110.8%

476

111.2%

974

104.5%

3,218

99.8%

3,278

100.5%

4,104

112.3%

4,758

100.5%

806

103.7%

U/U/U

379

129.7%

237

148.7%

63

76.9%

111

83.7%

192

70.7%

142

51.3%

224

101.4%

1,231

118.9%

10,208

117.0%

AA_Grp

Dur_Grp: 1-1

Dur_Grp: 2-2

Dur_Grp: 3-3

Dur_Grp: 4-5

Dur_Grp: 6-10

Dur_Grp: 11-15

Dur_Grp: 16-20

Dur_Grp: 21-30

Dur_Grp: 31-100

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

0-17

29

65.7%

15

53.7%

12

59.1%

17

73.1%

42

102.3%

27

70.5%

23

115.9%

18-25

525

97.5%

251

89.1%

184

112.3%

221

92.4%

167

103.2%

38

90.8%

71

81.9%

42

86.0%

26-30

372

106.2%

410

89.3%

367

134.1%

565

124.1%

498

111.0%

73

132.3%

13

80.3%

26

111.7%

0

0.0%

31-35

389

94.4%

424

99.5%

441

103.1%

788

107.0%

1,461

105.2%

410

121.8%

86

124.6%

19

128.3%

7

153.3%

36-40

399

131.0%

525

114.9%

593

105.3%

1,135

92.5%

2,691

95.7%

1,410

88.3%

370

112.1%

62

105.4%

14

54.0%

41-45

531

116.9%

642

96.1%

733

104.9%

1,659

97.7%

4,702

98.4%

3,326

99.1%

1,471

103.7%

247

102.9%

15

66.5%

46-50

609

78.3%

773

102.6%

950

98.6%

2,194

104.9%

6,546

100.0%

6,097

100.5%

3,360

100.4%

1,029

95.3%

80

145.6%

51-55

696

114.7%

1,037

99.6%

1,262

103.2%

2,693

95.6%

8,778

101.3%

8,866

100.7%

5,758

100.7%

2,843

89.3%

274

112.9%

56-60

668

96.1%

1,067

100.6%

1,399

90.9%

3,123

101.8%

10,254

101.3%

9,999

98.5%

6,943

99.0%

4,603

96.9%

927

120.5%

61-65

541

74.2%

849

115.4%

1,182

97.5%

2,968

105.2%

10,706

98.4%

11,195

99.1%

6,915

96.7%

5,041

106.5%

1,964

114.9%

66-70

368

117.5%

586

102.5%

852

100.2%

2,142

94.3%

8,996

98.1%

10,589

99.4%

6,086

100.0%

3,946

107.3%

1,932

117.7%

71-75

163

84.6%

284

91.7%

384

82.0%

1,213

88.5%

6,068

101.6%

7,575

97.8%

4,473

100.0%

2,334

115.6%

1,771

125.0%

76-80

158

105.8%

148

92.6%

170

140.9%

402

81.5%

3,179

100.3%

3,852

107.2%

2,627

97.9%

1,494

122.4%

3,014

101.3%

81-85

27

152.0%

114

125.4%

140

154.3%

256

86.6%

1,001

93.4%

1,108

123.3%

892

90.1%

1,190

118.8%

2,300

109.2%

86-90

22

226.3%

11

114.0%

19

107.0%

36

102.9%

249

121.7%

195

124.5%

388

111.8%

825

109.6%

698

100.0%

91-95

0

0.0%

5

51.6%

4

139.8%

7

43.9%

27

58.7%

38

79.5%

67

48.7%

285

123.3%

300

102.2%

96-100

0

0.0%

0

0.0%

0

0.0%

1

126.3%

3

81.1%

1

197.3%

4

381.2%

19

19.3%

50

56.6%

101-105

0

0.0%

1

53.1%

106-110

0

0.0%

111-115

0

0.0%

For the bivariate fit checks, the fit can be great in some cases and poor in others. This could be due to a need for higher level interactions.

7.2 PLT Only

Code
dat.term.plt <- readRDS("dat.term.plt.rds")
dat.term.plt <- dat.term.plt[Policies_Exposed > 0]

bUseAllInteractions <- FALSE

AA.brks <- c(-1,17,
             seq(25,115,5))
AA.lbls <- paste0(
  AA.brks[1:(length(AA.brks)-1)]+1,
  "-",
  AA.brks[2:length(AA.brks)]
)

Dur.brks <- c(0,1,2,3,5,10,15,20,30,100)
Dur.lbls <- paste0(
  Dur.brks[1:(length(Dur.brks)-1)]+1,
  "-",
  Dur.brks[2:length(Dur.brks)]
)

dat.term.plt[,`:=`(
  AA_Grp=cut(Attained_Age,
              breaks=AA.brks,
              labels=AA.lbls),
  Dur_Grp=cut(Duration,
               breaks=Dur.brks,
               labels=Dur.lbls)
)]



pred.cols <- names(dat.term.plt)[c(1,5,9,16:18)]

factor.cols <- c("Sex",
                 "Face_Amount_Band",
                 "SOA_Antp_Lvl_TP",
                 "UW",
                 "AA_Grp",
                 "Dur_Grp")

dat.term.plt[,
      (factor.cols):=lapply(.SD,factor),
      .SDcols=factor.cols]

UW.levels<-data.table(UW.levels=dat.term.plt[,levels(UW)])
UW.levels[,c("NS","Pref_Class","NClasses"):=tstrsplit(UW.levels,"/")]
setkeyv(UW.levels,c("NS","NClasses","Pref_Class"))

set.seed(traintest.seed)
dat.term.plt %>%
  mutate(
    IsTraining = (runif(nrow(.)) < training.fraction),
    Noise = rnorm(nrow(.)),
    .before = 1
  ) %>% 
  mutate(Face_Amount_Band=fct_relabel(
    Face_Amount_Band,
    #function(.) sub(":"," -",.,fixed=T)
    function(.) fa.remap[Face_Amount_Band.Old==.,Face_Amount_Band.New]
    ),
    AE_Count=Death_Count/ExpDth_Cnt_VBT2015,
    AE_Amount=Death_Claim_Amount/ExpDth_Amt_VBT2015,
    UW=factor(UW,levels=UW.levels$UW.levels)
  ) -> 
  dat.term.plt

gbm.pred.cols <- c("Noise",pred.cols)

Here is a data preview.

Code
dat.term.plt %>%
  head(10) %>%
  flextable()

IsTraining

Noise

Sex

Smoker_Status

Attained_Age

Duration

Face_Amount_Band

SOA_Post_Lvl_Ind

Number_of_Pfd_Classes

Preferred_Class

SOA_Antp_Lvl_TP

Death_Count

Policies_Exposed

ExpDth_Cnt_VBT2015

Death_Claim_Amount

Amount_Exposed

ExpDth_Amt_VBT2015

UW

AA_Grp

Dur_Grp

AE_Count

AE_Amount

TRUE

0.1278386

F

NS

55

8

500K - <1M

PLT

U

U

Unknown

0

0.796549

0.001099238

0

398,274.5

549.6188

NS/U/U

51-55

6-10

0

0

TRUE

-0.1518745

F

NS

56

9

50K - <100K

PLT

2

2

Unknown

0

8.847863

0.013624558

0

491,198.0

756.3875

NS/2/2

56-60

6-10

0

0

TRUE

-0.0547556

F

NS

56

9

100K - <250K

PLT

3

1

Unknown

0

2.627397

0.003993918

0

262,739.7

399.3917

NS/1/3

56-60

6-10

0

0

TRUE

-1.6323762

F

NS

56

9

100K - <250K

PLT

U

U

Unknown

0

21.468803

0.033037792

0

2,438,239.4

3,752.4723

NS/U/U

56-60

6-10

0

0

TRUE

1.5316881

F

NS

56

9

100K - <250K

PLT

3

3

Unknown

0

33.395670

0.049664452

0

4,304,540.0

6,412.5258

NS/3/3

56-60

6-10

0

0

TRUE

-1.7326077

F

NS

56

9

250K - <500K

PLT

3

3

Unknown

0

15.885403

0.023630075

0

4,442,986.5

6,606.8924

NS/3/3

56-60

6-10

0

0

FALSE

-0.8680619

F

NS

56

9

250K - <500K

PLT

3

1

Unknown

0

2.824276

0.004231874

0

733,773.9

1,098.9717

NS/1/3

56-60

6-10

0

0

TRUE

1.0450097

F

NS

57

10

50K - <100K

PLT

2

2

Unknown

0

16.856679

0.029410194

0

888,904.2

1,551.5934

NS/2/2

56-60

6-10

0

0

TRUE

-0.6840765

F

NS

57

10

100K - <250K

PLT

2

1

Unknown

0

10.063515

0.017114580

0

1,218,666.6

2,070.2707

NS/1/2

56-60

6-10

0

0

TRUE

-1.8073461

F

NS

57

10

100K - <250K

PLT

3

1

Unknown

0

5.295008

0.009276088

0

614,610.4

1,077.4017

NS/1/3

56-60

6-10

0

0

7.2.1 Feature Discovery with Catboost

As in the predictive analytics framework, gradient boosted machines as implemented with Catboost are used for exploratory data analysis. The framework uses LightGBM, while Catboost is used here. It is easier to use Catboost with GPUs.

Code
dat.term.plt %>% 
  filter(IsTraining == TRUE) %>% 
  select(all_of(gbm.pred.cols)) %>% 
  mutate(Sex = as.numeric(Sex)) %>%
  catboost.load_pool(
    data=.,
    label = dat.term.plt[IsTraining==TRUE,AE_Amount],
    weight = dat.term.plt[IsTraining==TRUE,ExpDth_Amt_VBT2015]) ->
  train_pool_plt

dat.term.plt %>% 
  filter(IsTraining == FALSE) %>% 
  select(all_of(gbm.pred.cols)) %>% 
  mutate(Sex = as.numeric(Sex)) %>%
  catboost.load_pool(
    data=.,
    label = dat.term.plt[IsTraining==FALSE,AE_Amount],
    weight = dat.term.plt[IsTraining==FALSE,ExpDth_Amt_VBT2015]) ->
  test_pool_plt

set.seed(catboost.seed)
if(bUseCache & file.exists(
  'dat.term.mod.plt.cbm'
) & !bInvalidateCaches)
{
  cb.model.plt <- catboost.load_model(model_path = 'dat.term.mod.plt.cbm')
} else {
  cb.model.plt <- catboost.train(learn_pool=train_pool_plt,
                             test_pool = test_pool_plt,
                             params=fit_params)
  
  catboost.save_model(cb.model.plt,
                      model_path = 'dat.term.mod.plt.cbm')
}

7.2.2 Feature Importance

Next is to check for the variables with most variability. Attained age and underwriting are prominent.

Code
shp.plt <- shapviz(
  cb.model.plt,
  X=as.data.frame(dat.term.plt[IsTraining==TRUE,..gbm.pred.cols]),
  X_pred=train_pool_plt
)
setDT(shp.plt$X)
shp.plt$X[,c("NS","Pref_Class","NClasses"):=tstrsplit(UW,"/")]

sv_importance(shp.plt) + theme_minimal()

7.2.3 Feature Interactions

The top three interactions are face amount band with each of underwriting, attained age, and level term period.

Code
imp.int.plt <- catboost.get_feature_importance(
  cb.model.plt,
  type="Interaction"
)



mod.ft.map.plt <- data.table(
  FeatureNames=names(dat.term.plt[IsTraining==TRUE,..gbm.pred.cols])
)

mod.ft.map.plt[,ID:=1:nrow(.SD)-1]

imp.int.plt %>%
  merge(mod.ft.map.plt,
        by.x="feature1_index",
        by.y="ID") %>%
  merge(mod.ft.map.plt,
        by.x="feature2_index",
        by.y="ID") %>%
  setnames(
    old=c("FeatureNames.x","FeatureNames.y"),
    new=c("Feature1","Feature2")
  ) %>% 
  mutate(feature1_index=NULL,feature2_index=NULL) %>%
  as.data.table() %>%
  setcolorder(c("Feature1","Feature2","score")) -> 
  imp.int.plt

imp.int.plt[order(-score)] %>%
  filter(Feature1 != "Noise") %>%
   flextable() %>%
  colformat_double(j="score",digits=2)

Feature1

Feature2

score

Face_Amount_Band

UW

15.75

Sex

Face_Amount_Band

9.68

Face_Amount_Band

AA_Grp

9.31

Face_Amount_Band

SOA_Antp_Lvl_TP

6.10

Sex

UW

4.64

Face_Amount_Band

Dur_Grp

4.46

Sex

AA_Grp

4.21

UW

AA_Grp

4.08

SOA_Antp_Lvl_TP

UW

2.95

Sex

SOA_Antp_Lvl_TP

2.70

SOA_Antp_Lvl_TP

AA_Grp

1.84

UW

Dur_Grp

1.40

Sex

Dur_Grp

1.01

SOA_Antp_Lvl_TP

Dur_Grp

0.72

AA_Grp

Dur_Grp

0.46

7.2.4 Feature Interaction Plots

Each of the interactions are discussed below with plots using SHAP values from both sides of the interaction.

7.2.4.1 UW

7.2.4.1.1 Face Amount Band SHAPs

To the extent that an interaction is present, it appears to be occurring at face amounts 2.5 million and above.

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.plt$S[,"Face_Amount_Band"]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for non-smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.plt$S[,"Face_Amount_Band"]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '2') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for non-smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.plt$S[,"Face_Amount_Band"]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '3') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for non-smokers, 3 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.plt$S[,"Face_Amount_Band"]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '4') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for non-smokers, 4 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.plt$S[,"Face_Amount_Band"]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.plt$S[,"Face_Amount_Band"]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == '2') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.plt$S[,"Face_Amount_Band"]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for unismokers")) +
  theme_minimal()

7.2.4.1.2 UW SHAPs

On the UW SHAP side, there appears to be little to no interaction, except at face amounts 2.5 million and above.

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.plt$S[,"UW"]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.plt$S[,"UW"]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '2') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.plt$S[,"UW"]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '3') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, 3 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.plt$S[,"UW"]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '4') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, 4 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.plt$S[,"UW"]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.plt$S[,"UW"]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == '2') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.plt$S[,"UW"]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for unismokers")) +
  theme_minimal()

7.2.4.2 Attained Age Group

7.2.4.2.1 Face Amount Band SHAPs

For face amount SHAPS by attained age, any interaction is once again a high face phenomenon, if it indeed exists. One can see that for the 10M+ band at least, there is a cluster of nearly 0% factors, suggesting that there may be some challenges around sparse data. Additionally, for the 5M-<10M face amounts, there is some evidence of multi-modality across attained ages. Again, this may be due to thin data.

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                 AA_Grp)],
    shap=shp.plt$S[,"Face_Amount_Band"]
  )
) %>%
  ggplot(aes(y=AA_Grp, x=exp(shap),color=Face_Amount_Band,fill=Face_Amount_Band)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,2)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("by attained age")) +
  theme_minimal()

7.2.4.2.2 Attained Age SHAPs

On the other hand, the attained age SHAPs do exhibit a clear interaction with face amount band.

In the first plot, it is clear that there is a separation in SHAP value distributions with higher attained ages separating from lower attained ages. The approximate breakpoint is most evident in the second plot, at about attained age 66+ or so.

The other interesting phenomenon is the shifts in the SHAP distributions for each of the face amount bands of 5M - < 10M and 10M+. In the second plot, if one’s eyes follows the distributions from bottom to top, one sees that starting at attained age group 18-25, the distribution starts low, shifts rightward through about age group 61-65, then shifts leftward again through age group 81-85. Given the claim count, this is arguably weak evidence for anti-selection at the high face amounts for middle attained ages.

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    AA_Grp)],
    shap=shp.plt$S[,"AA_Grp"]
  )
) %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=AA_Grp,fill=AA_Grp)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,2)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect AA_Grp"),
          subtitle = paste0("by face amount band")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                 AA_Grp)],
    shap=shp.plt$S[,"AA_Grp"]
  )
) %>%
  ggplot(aes(y=AA_Grp, x=exp(shap),color=Face_Amount_Band,fill=Face_Amount_Band)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,2)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect AA_Grp"),
          subtitle = paste0("by attained age")) +
  theme_minimal()

7.2.4.3 Sex and Face Amount Band

7.2.4.3.1 Face Amount Band SHAPs

As with the other groups, the interaction, if any, is a high face amount phenomenon.

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                 Sex)],
    shap=shp.plt$S[,"Face_Amount_Band"]
  )
) %>%
  ggplot(aes(y=Sex, x=exp(shap),color=Face_Amount_Band,fill=Face_Amount_Band)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,2)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("by sex")) +
  theme_minimal()

7.2.4.3.2 Sex SHAPs

There appears to be an interaction at the highest face amounts for Sex.

Code
data.table(
  cbind(
    shp.plt$X[,.(Face_Amount_Band,
                    Sex)],
    shap=shp.plt$S[,"Sex"]
  )
) %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Sex,fill=Sex)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,2)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Sex"),
          subtitle = paste0("by face amount band")) +
  theme_minimal()

7.2.5 Next Steps in Modeling

The catboost modeling is informative here for exploratory analytics. We next build an elastic net model which includes the main effects plus the top eight interactions from the catboost analysis. This cutoff is chosen arbitrarily. The top three are stronger than the others, while the next five seem to cluster in the middle. These interactions may turn out to be unneeded. Elastic net modeling will reveal which is truly important in a mortality model.

7.2.6 Elastic Net Models

The following formula is used for the PLT model.

Code
if(bUseAllInteractions) {
  glmnetFormula <- as.formula(
    paste0(
      "~ -1 + (",
      paste(pred.cols,collapse=" + "),
      ")^2"
    )
)
} else {
  glmnetFormula <- as.formula(
    paste(c("~ -1",
            pred.cols,
            imp.int.plt[order(-score)][Feature1 != "Noise",
                           paste0(Feature1,":",Feature2)]),
          collapse=" + ")
  )
}


print(glmnetFormula)
~-1 + Sex + Face_Amount_Band + SOA_Antp_Lvl_TP + UW + AA_Grp + 
    Dur_Grp + Face_Amount_Band:UW + Sex:Face_Amount_Band + Face_Amount_Band:AA_Grp + 
    Face_Amount_Band:SOA_Antp_Lvl_TP + Sex:UW + Face_Amount_Band:Dur_Grp + 
    Sex:AA_Grp + UW:AA_Grp + SOA_Antp_Lvl_TP:UW + Sex:SOA_Antp_Lvl_TP + 
    SOA_Antp_Lvl_TP:AA_Grp + UW:Dur_Grp + Sex:Dur_Grp + SOA_Antp_Lvl_TP:Dur_Grp + 
    AA_Grp:Dur_Grp

The interactions included in the model are as follows:

Code
data.table(Interaction=imp.int.plt[order(-score)][Feature1 != "Noise",
                         paste0(Feature1,":",Feature2)]) %>%
  flextable()

Interaction

Face_Amount_Band:UW

Sex:Face_Amount_Band

Face_Amount_Band:AA_Grp

Face_Amount_Band:SOA_Antp_Lvl_TP

Sex:UW

Face_Amount_Band:Dur_Grp

Sex:AA_Grp

UW:AA_Grp

SOA_Antp_Lvl_TP:UW

Sex:SOA_Antp_Lvl_TP

SOA_Antp_Lvl_TP:AA_Grp

UW:Dur_Grp

Sex:Dur_Grp

SOA_Antp_Lvl_TP:Dur_Grp

AA_Grp:Dur_Grp

7.2.6.1 Model Fit

Code
dat.term.el.plt <- prepELData(
  formula=glmnetFormula,
  data=dat.term.plt,
  predictors = pred.cols,
  response = "AE_Amount",
  weights = "ExpDth_Amt_VBT2015",
  useSparse = T
)


set.seed(cvfit.seed)
if(bUseCache & file.exists(
  'term.model.el.plt.rds'
) & !bInvalidateCaches)
{
  cvfit.term.plt <- readRDS('term.model.el.plt.rds')
} else
{
  cvfit.term.plt <- fitCVGLMNet(
    dat.term.el.plt,
    nfolds = nFolds
  )
  
  if(bUseCache)
    saveRDS(cvfit.term.plt, 'term.model.el.plt.rds')
}
7.2.6.1.1 Usual plots for Elastic Net Models

When presenting elastic net models, the cross validation plot for \(\lambda\) and the coefficient shrinkage plots are provided.

At the minimum \(\lambda\) of 0.002387, the model has 266 parameters.

Code
plot(cvfit.term.plt)

Code
plot(cvfit.term.plt$glmnet.fit,xvar="lambda")

7.2.6.1.2 Factor Table

The exponentiated coefficients are as follows:

Code
reformatCoefs(cvfit.term.plt, pred.cols)  %>%
  filter(Coef != 0) %>%
  select(Feature1Name,
         Feature1Level,
         Feature2Name,
         Feature2Level,
         Coef) %>%
  mutate(Coef=exp(Coef)) %>%
  flextable() %>%
  set_formatter(
    Coef=function(x) paste0(sprintf("%.01f", 100*x),"%")
  ) %>%
  theme_vanilla()

Feature1Name

Feature1Level

Feature2Name

Feature2Level

Coef

(Intercept)

(Intercept)

104.2%

Sex

F

101.6%

Sex

M

99.8%

Face_Amount_Band

10K - <25K

97.3%

Face_Amount_Band

100K - <250K

97.0%

Face_Amount_Band

250K - <500K

97.3%

Face_Amount_Band

2.5M - <5M

119.7%

Face_Amount_Band

10M+

138.4%

SOA_Antp_Lvl_TP

10 yr anticipated

113.5%

SOA_Antp_Lvl_TP

15 yr anticipated

107.6%

SOA_Antp_Lvl_TP

20 yr anticipated

114.2%

SOA_Antp_Lvl_TP

Unknown

101.5%

UW

NS/2/2

118.7%

UW

NS/1/3

96.0%

UW

NS/2/3

103.8%

UW

NS/3/3

100.8%

UW

NS/3/4

123.4%

UW

NS/4/4

144.6%

UW

S/1/2

87.3%

UW

S/2/2

110.4%

UW

S/U/U

109.2%

UW

U/U/U

112.6%

AA_Grp

31-35

92.6%

AA_Grp

36-40

96.0%

AA_Grp

41-45

87.1%

AA_Grp

46-50

97.2%

AA_Grp

66-70

101.4%

AA_Grp

71-75

107.4%

AA_Grp

76-80

133.0%

AA_Grp

81-85

127.4%

Dur_Grp

11-15

106.5%

Dur_Grp

16-20

109.3%

Face_Amount_Band

10K - <25K

UW

NS/2/2

96.0%

Face_Amount_Band

50K - <100K

UW

NS/2/2

110.0%

Face_Amount_Band

100K - <250K

UW

NS/2/2

108.5%

Face_Amount_Band

250K - <500K

UW

NS/2/2

102.4%

Face_Amount_Band

1M - <2.5M

UW

NS/2/2

87.5%

Face_Amount_Band

500K - <1M

UW

NS/1/3

102.8%

Face_Amount_Band

2.5M - <5M

UW

NS/1/3

151.8%

Face_Amount_Band

100K - <250K

UW

NS/2/3

94.8%

Face_Amount_Band

1M - <2.5M

UW

NS/2/3

100.1%

Face_Amount_Band

2.5M - <5M

UW

NS/2/3

125.3%

Face_Amount_Band

100K - <250K

UW

NS/3/3

95.6%

Face_Amount_Band

500K - <1M

UW

NS/3/3

89.8%

Face_Amount_Band

1M - <2.5M

UW

NS/3/3

120.9%

Face_Amount_Band

2.5M - <5M

UW

NS/3/3

65.0%

Face_Amount_Band

5M - <10M

UW

NS/3/3

228.9%

Face_Amount_Band

10M+

UW

NS/3/3

433.0%

Face_Amount_Band

500K - <1M

UW

NS/1/4

127.6%

Face_Amount_Band

1M - <2.5M

UW

NS/1/4

145.4%

Face_Amount_Band

2.5M - <5M

UW

NS/1/4

83.7%

Face_Amount_Band

500K - <1M

UW

NS/2/4

102.4%

Face_Amount_Band

2.5M - <5M

UW

NS/2/4

100.4%

Face_Amount_Band

5M - <10M

UW

NS/2/4

69.4%

Face_Amount_Band

100K - <250K

UW

NS/3/4

92.0%

Face_Amount_Band

1M - <2.5M

UW

NS/3/4

109.2%

Face_Amount_Band

2.5M - <5M

UW

NS/3/4

165.2%

Face_Amount_Band

5M - <10M

UW

NS/3/4

158.4%

Face_Amount_Band

250K - <500K

UW

NS/4/4

100.8%

Face_Amount_Band

500K - <1M

UW

NS/4/4

114.7%

Face_Amount_Band

25K - <50K

UW

NS/U/U

107.8%

Face_Amount_Band

50K - <100K

UW

NS/U/U

114.8%

Face_Amount_Band

500K - <1M

UW

NS/U/U

98.4%

Face_Amount_Band

2.5M - <5M

UW

NS/U/U

138.9%

Face_Amount_Band

100K - <250K

UW

S/1/2

114.8%

Face_Amount_Band

100K - <250K

UW

S/2/2

110.8%

Face_Amount_Band

250K - <500K

UW

S/2/2

105.7%

Face_Amount_Band

100K - <250K

UW

S/U/U

98.3%

Face_Amount_Band

5M - <10M

UW

S/U/U

123.4%

Face_Amount_Band

10M+

UW

S/U/U

774.6%

Sex

M

Face_Amount_Band

25K - <50K

103.5%

Sex

M

Face_Amount_Band

50K - <100K

105.8%

Sex

M

Face_Amount_Band

100K - <250K

100.8%

Sex

M

Face_Amount_Band

2.5M - <5M

117.2%

Sex

M

Face_Amount_Band

5M - <10M

81.1%

Sex

M

Face_Amount_Band

10M+

25.6%

Face_Amount_Band

1M - <2.5M

AA_Grp

41-45

109.9%

Face_Amount_Band

2.5M - <5M

AA_Grp

41-45

88.2%

Face_Amount_Band

250K - <500K

AA_Grp

46-50

93.0%

Face_Amount_Band

500K - <1M

AA_Grp

46-50

98.8%

Face_Amount_Band

1M - <2.5M

AA_Grp

46-50

93.1%

Face_Amount_Band

5M - <10M

AA_Grp

46-50

109.2%

Face_Amount_Band

25K - <50K

AA_Grp

51-55

100.9%

Face_Amount_Band

50K - <100K

AA_Grp

51-55

105.4%

Face_Amount_Band

250K - <500K

AA_Grp

51-55

93.0%

Face_Amount_Band

500K - <1M

AA_Grp

51-55

96.3%

Face_Amount_Band

2.5M - <5M

AA_Grp

51-55

128.0%

Face_Amount_Band

50K - <100K

AA_Grp

56-60

106.6%

Face_Amount_Band

100K - <250K

AA_Grp

56-60

97.9%

Face_Amount_Band

250K - <500K

AA_Grp

56-60

96.4%

Face_Amount_Band

500K - <1M

AA_Grp

56-60

94.9%

Face_Amount_Band

1M - <2.5M

AA_Grp

56-60

91.7%

Face_Amount_Band

5M - <10M

AA_Grp

56-60

210.2%

Face_Amount_Band

10M+

AA_Grp

56-60

196.4%

Face_Amount_Band

100K - <250K

AA_Grp

61-65

99.2%

Face_Amount_Band

1M - <2.5M

AA_Grp

61-65

90.7%

Face_Amount_Band

5M - <10M

AA_Grp

61-65

52.9%

Face_Amount_Band

10M+

AA_Grp

61-65

447.9%

Face_Amount_Band

100K - <250K

AA_Grp

66-70

101.0%

Face_Amount_Band

250K - <500K

AA_Grp

66-70

100.3%

Face_Amount_Band

500K - <1M

AA_Grp

66-70

108.6%

Face_Amount_Band

1M - <2.5M

AA_Grp

66-70

103.0%

Face_Amount_Band

2.5M - <5M

AA_Grp

66-70

64.4%

Face_Amount_Band

5M - <10M

AA_Grp

66-70

99.1%

Face_Amount_Band

10M+

AA_Grp

66-70

97.8%

Face_Amount_Band

50K - <100K

AA_Grp

71-75

88.0%

Face_Amount_Band

500K - <1M

AA_Grp

71-75

116.1%

Face_Amount_Band

1M - <2.5M

AA_Grp

71-75

107.2%

Face_Amount_Band

5M - <10M

AA_Grp

71-75

100.4%

Face_Amount_Band

25K - <50K

AA_Grp

76-80

89.1%

Face_Amount_Band

50K - <100K

AA_Grp

76-80

88.5%

Face_Amount_Band

250K - <500K

AA_Grp

76-80

116.1%

Face_Amount_Band

500K - <1M

AA_Grp

76-80

111.3%

Face_Amount_Band

100K - <250K

AA_Grp

81-85

106.6%

Face_Amount_Band

100K - <250K

SOA_Antp_Lvl_TP

10 yr anticipated

98.8%

Face_Amount_Band

5M - <10M

SOA_Antp_Lvl_TP

10 yr anticipated

131.2%

Face_Amount_Band

10M+

SOA_Antp_Lvl_TP

10 yr anticipated

112.0%

Face_Amount_Band

50K - <100K

SOA_Antp_Lvl_TP

15 yr anticipated

97.4%

Face_Amount_Band

100K - <250K

SOA_Antp_Lvl_TP

15 yr anticipated

94.7%

Face_Amount_Band

500K - <1M

SOA_Antp_Lvl_TP

15 yr anticipated

115.0%

Face_Amount_Band

1M - <2.5M

SOA_Antp_Lvl_TP

15 yr anticipated

110.3%

Face_Amount_Band

2.5M - <5M

SOA_Antp_Lvl_TP

15 yr anticipated

104.1%

Face_Amount_Band

5M - <10M

SOA_Antp_Lvl_TP

15 yr anticipated

99.7%

Face_Amount_Band

10M+

SOA_Antp_Lvl_TP

15 yr anticipated

234.5%

Face_Amount_Band

100K - <250K

SOA_Antp_Lvl_TP

Unknown

95.8%

Face_Amount_Band

500K - <1M

SOA_Antp_Lvl_TP

Unknown

98.7%

Face_Amount_Band

1M - <2.5M

SOA_Antp_Lvl_TP

Unknown

101.4%

Face_Amount_Band

2.5M - <5M

SOA_Antp_Lvl_TP

Unknown

146.5%

Face_Amount_Band

5M - <10M

SOA_Antp_Lvl_TP

Unknown

111.9%

Sex

M

UW

NS/2/2

101.2%

Sex

M

UW

NS/3/3

104.1%

Sex

M

UW

NS/1/4

91.3%

Sex

M

UW

NS/2/4

123.0%

Sex

M

UW

NS/U/U

108.0%

Sex

M

UW

S/1/2

95.8%

Sex

M

UW

S/U/U

97.7%

Sex

M

UW

U/U/U

116.6%

Face_Amount_Band

50K - <100K

Dur_Grp

11-15

91.9%

Face_Amount_Band

100K - <250K

Dur_Grp

11-15

96.8%

Face_Amount_Band

2.5M - <5M

Dur_Grp

11-15

100.1%

Face_Amount_Band

5M - <10M

Dur_Grp

11-15

133.2%

Face_Amount_Band

50K - <100K

Dur_Grp

16-20

99.8%

Face_Amount_Band

100K - <250K

Dur_Grp

16-20

96.4%

Face_Amount_Band

250K - <500K

Dur_Grp

16-20

106.3%

Face_Amount_Band

1M - <2.5M

Dur_Grp

16-20

105.3%

Face_Amount_Band

10M+

Dur_Grp

16-20

141.0%

Face_Amount_Band

100K - <250K

Dur_Grp

21-30

105.3%

Face_Amount_Band

500K - <1M

Dur_Grp

21-30

105.6%

Face_Amount_Band

1M - <2.5M

Dur_Grp

21-30

97.0%

Face_Amount_Band

5M - <10M

Dur_Grp

21-30

85.3%

Sex

M

AA_Grp

51-55

98.0%

Sex

M

AA_Grp

56-60

101.5%

Sex

M

AA_Grp

71-75

103.4%

UW

NS/2/3

AA_Grp

36-40

109.9%

UW

NS/2/2

AA_Grp

41-45

116.9%

UW

NS/U/U

AA_Grp

41-45

123.3%

UW

NS/2/3

AA_Grp

46-50

88.1%

UW

NS/3/4

AA_Grp

46-50

104.6%

UW

NS/2/2

AA_Grp

51-55

104.1%

UW

NS/1/3

AA_Grp

51-55

94.6%

UW

NS/2/2

AA_Grp

56-60

100.6%

UW

NS/2/3

AA_Grp

56-60

105.4%

UW

NS/3/3

AA_Grp

56-60

104.4%

UW

NS/1/4

AA_Grp

56-60

103.4%

UW

NS/3/4

AA_Grp

56-60

118.0%

UW

NS/U/U

AA_Grp

56-60

94.6%

UW

S/U/U

AA_Grp

56-60

116.6%

UW

NS/3/3

AA_Grp

61-65

120.9%

UW

NS/1/4

AA_Grp

61-65

93.0%

UW

NS/4/4

AA_Grp

61-65

102.1%

UW

NS/U/U

AA_Grp

61-65

95.9%

UW

NS/3/3

AA_Grp

66-70

87.9%

UW

NS/1/3

AA_Grp

71-75

67.5%

UW

NS/2/3

AA_Grp

71-75

96.6%

UW

NS/1/4

AA_Grp

71-75

111.6%

UW

NS/U/U

AA_Grp

71-75

125.6%

UW

S/U/U

AA_Grp

71-75

103.0%

UW

NS/2/2

AA_Grp

76-80

89.4%

UW

NS/1/3

AA_Grp

76-80

101.7%

UW

NS/3/3

AA_Grp

76-80

85.6%

UW

NS/2/4

AA_Grp

76-80

124.6%

UW

NS/U/U

AA_Grp

76-80

110.8%

UW

NS/2/2

AA_Grp

81-85

87.5%

UW

NS/3/3

AA_Grp

81-85

101.8%

UW

NS/U/U

AA_Grp

81-85

106.3%

UW

NS/2/2

AA_Grp

86-90

99.8%

SOA_Antp_Lvl_TP

10 yr anticipated

UW

NS/2/2

101.9%

SOA_Antp_Lvl_TP

15 yr anticipated

UW

NS/2/2

97.6%

SOA_Antp_Lvl_TP

20 yr anticipated

UW

NS/2/2

105.6%

SOA_Antp_Lvl_TP

Unknown

UW

NS/2/2

99.1%

SOA_Antp_Lvl_TP

Unknown

UW

NS/1/3

120.7%

SOA_Antp_Lvl_TP

10 yr anticipated

UW

NS/2/3

97.4%

SOA_Antp_Lvl_TP

15 yr anticipated

UW

NS/2/3

97.8%

SOA_Antp_Lvl_TP

Unknown

UW

NS/2/3

115.6%

SOA_Antp_Lvl_TP

10 yr anticipated

UW

NS/3/3

120.5%

SOA_Antp_Lvl_TP

15 yr anticipated

UW

NS/3/3

133.1%

SOA_Antp_Lvl_TP

Unknown

UW

NS/3/3

96.2%

SOA_Antp_Lvl_TP

10 yr anticipated

UW

NS/1/4

115.4%

SOA_Antp_Lvl_TP

15 yr anticipated

UW

NS/1/4

87.2%

SOA_Antp_Lvl_TP

10 yr anticipated

UW

NS/2/4

116.2%

SOA_Antp_Lvl_TP

15 yr anticipated

UW

NS/2/4

87.4%

SOA_Antp_Lvl_TP

Unknown

UW

NS/3/4

136.1%

SOA_Antp_Lvl_TP

10 yr anticipated

UW

NS/4/4

107.4%

SOA_Antp_Lvl_TP

10 yr anticipated

UW

NS/U/U

77.7%

SOA_Antp_Lvl_TP

15 yr anticipated

UW

NS/U/U

138.4%

SOA_Antp_Lvl_TP

20 yr anticipated

UW

NS/U/U

86.5%

SOA_Antp_Lvl_TP

Unknown

UW

NS/U/U

94.2%

SOA_Antp_Lvl_TP

10 yr anticipated

UW

S/1/2

105.2%

SOA_Antp_Lvl_TP

Unknown

UW

S/U/U

90.8%

Sex

M

SOA_Antp_Lvl_TP

15 yr anticipated

114.2%

Sex

M

SOA_Antp_Lvl_TP

20 yr anticipated

108.5%

Sex

M

SOA_Antp_Lvl_TP

Unknown

108.1%

SOA_Antp_Lvl_TP

10 yr anticipated

AA_Grp

41-45

86.4%

SOA_Antp_Lvl_TP

10 yr anticipated

AA_Grp

46-50

98.4%

SOA_Antp_Lvl_TP

20 yr anticipated

AA_Grp

51-55

97.4%

SOA_Antp_Lvl_TP

Unknown

AA_Grp

56-60

103.7%

SOA_Antp_Lvl_TP

10 yr anticipated

AA_Grp

61-65

101.7%

SOA_Antp_Lvl_TP

20 yr anticipated

AA_Grp

61-65

106.8%

SOA_Antp_Lvl_TP

15 yr anticipated

AA_Grp

66-70

112.8%

SOA_Antp_Lvl_TP

20 yr anticipated

AA_Grp

66-70

108.2%

SOA_Antp_Lvl_TP

10 yr anticipated

AA_Grp

71-75

113.7%

SOA_Antp_Lvl_TP

15 yr anticipated

AA_Grp

71-75

116.5%

SOA_Antp_Lvl_TP

20 yr anticipated

AA_Grp

71-75

108.2%

SOA_Antp_Lvl_TP

15 yr anticipated

AA_Grp

76-80

106.4%

SOA_Antp_Lvl_TP

Unknown

AA_Grp

76-80

124.2%

SOA_Antp_Lvl_TP

15 yr anticipated

AA_Grp

81-85

129.0%

SOA_Antp_Lvl_TP

20 yr anticipated

AA_Grp

81-85

86.0%

UW

NS/3/3

Dur_Grp

11-15

116.4%

UW

NS/3/4

Dur_Grp

11-15

155.5%

UW

NS/4/4

Dur_Grp

11-15

115.7%

UW

NS/U/U

Dur_Grp

11-15

103.2%

UW

S/1/2

Dur_Grp

11-15

101.2%

UW

NS/2/3

Dur_Grp

16-20

92.5%

UW

NS/3/3

Dur_Grp

16-20

100.2%

UW

S/1/2

Dur_Grp

16-20

117.0%

UW

S/U/U

Dur_Grp

16-20

111.4%

UW

NS/U/U

Dur_Grp

21-30

99.1%

UW

S/1/2

Dur_Grp

21-30

103.4%

UW

S/U/U

Dur_Grp

21-30

101.8%

UW

U/U/U

Dur_Grp

21-30

103.9%

Sex

M

Dur_Grp

16-20

102.9%

Sex

M

Dur_Grp

21-30

107.4%

Sex

M

Dur_Grp

31-100

100.3%

SOA_Antp_Lvl_TP

10 yr anticipated

Dur_Grp

11-15

132.6%

SOA_Antp_Lvl_TP

Unknown

Dur_Grp

11-15

123.6%

SOA_Antp_Lvl_TP

15 yr anticipated

Dur_Grp

16-20

131.3%

SOA_Antp_Lvl_TP

Unknown

Dur_Grp

16-20

106.9%

SOA_Antp_Lvl_TP

15 yr anticipated

Dur_Grp

21-30

87.5%

SOA_Antp_Lvl_TP

20 yr anticipated

Dur_Grp

21-30

101.7%

SOA_Antp_Lvl_TP

Unknown

Dur_Grp

21-30

106.9%

SOA_Antp_Lvl_TP

15 yr anticipated

Dur_Grp

31-100

106.1%

AA_Grp

36-40

Dur_Grp

11-15

82.0%

AA_Grp

41-45

Dur_Grp

11-15

91.3%

AA_Grp

56-60

Dur_Grp

11-15

97.8%

AA_Grp

76-80

Dur_Grp

11-15

114.2%

AA_Grp

81-85

Dur_Grp

11-15

102.9%

AA_Grp

46-50

Dur_Grp

16-20

93.0%

AA_Grp

56-60

Dur_Grp

16-20

98.8%

AA_Grp

61-65

Dur_Grp

16-20

98.5%

AA_Grp

66-70

Dur_Grp

16-20

106.1%

AA_Grp

71-75

Dur_Grp

16-20

106.5%

AA_Grp

81-85

Dur_Grp

16-20

102.4%

AA_Grp

51-55

Dur_Grp

21-30

97.0%

AA_Grp

61-65

Dur_Grp

21-30

108.5%

AA_Grp

66-70

Dur_Grp

21-30

117.6%

AA_Grp

71-75

Dur_Grp

21-30

107.8%

In the plots and tables that follow, we fix all of the other variables at their middle values when extracting final factors.

We generate some supporting tables: a factor grid for attaching to the experience, and a list of interactions present in the model

Code
dat.term.plt[,..pred.cols] %>%
  lapply(levels) %>%
  expand.grid() %>%
  setDT() -> 
  dat.term.plt.grid 

dat.term.plt.grid %>%
  model.Matrix(
    object=glmnetFormula,
    data=.,
    sparse=T
  ) %>%
  predict(
    cvfit.term.plt,
    newx=.,
    s="lambda.min"
  ) %>% 
  as.vector() ->
  newCoef

dat.term.plt.grid %>%
  add_column(
    Factor=exp(newCoef)
  )  %>% 
  setDT() -> 
  dat.term.plt.grid 

write.xlsx(dat.term.plt.grid,
           file="dat.term.plt.grid.xlsx")

reformatCoefs(cvfit.term.plt, pred.cols) %>%
  filter(Coef != 0 & !is.na(Feature2Name)) %>% 
  select(Feature1Name,Feature2Name) %>% 
  distinct() %>%
  as.list() %>%
  purrr::list_transpose() ->
  term.plt.int.list
7.2.6.1.3 Plots of Terms

Below are plots of the 2-way interaction terms, with external factors fixed at their middle values.

One thing to note in these plots is the upward sloping mortality from attained age 65 or so through attained age 85. Otherwise, effects can be subtle except as called out below.

  1. Face Amount Band x UW: There is a general increase in mortality with increasing face amount band above 2.5 million, with the increase more pronounced at certain UW combinations.
  2. Face Amount Band x UW: The increase above 2.5 million is more pronounced for females.
  3. Face Amount Band x Attained Age: The increases above 2.5 million appear predominantly in the ages 60-85.
  4. Face Amount Band x Term Length: The increase above 2.5 million is more pronounced for the 15-year term length.
  5. Duration x Term Length: The 10- and 15-year term lengths exhibit bumps in their first post-level durations.
7.2.6.1.4 Tables of Terms

Below are tables of the 2-way interaction terms, with external factors fixed at their middle values.

Face_Amount_Band

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

<10K

118.6%

138.3%

113.9%

127.1%

193.3%

107.1%

103.6%

268.6%

198.5%

160.4%

104.7%

131.0%

151.1%

133.6%

10K - <25K

115.5%

129.1%

110.8%

123.6%

188.1%

104.2%

100.9%

261.4%

193.1%

156.0%

101.9%

127.5%

147.1%

130.0%

25K - <50K

118.6%

138.3%

113.9%

127.1%

193.3%

107.1%

103.6%

268.6%

198.5%

172.8%

104.7%

131.0%

151.1%

133.6%

50K - <100K

113.2%

145.1%

108.6%

121.2%

184.4%

102.1%

98.9%

256.2%

189.3%

175.5%

99.9%

125.0%

144.2%

127.5%

100K - <250K

103.2%

130.4%

99.0%

104.8%

160.7%

93.1%

90.1%

214.8%

172.6%

139.4%

104.6%

126.2%

129.1%

116.2%

250K - <500K

111.3%

132.9%

106.8%

119.2%

181.4%

100.5%

97.3%

252.0%

187.8%

150.5%

98.3%

130.0%

141.8%

125.4%

500K - <1M

129.5%

150.9%

127.8%

138.7%

189.4%

149.1%

115.8%

293.1%

248.5%

172.3%

114.3%

143.0%

164.9%

145.8%

1M - <2.5M

120.1%

122.4%

115.2%

128.7%

236.5%

157.5%

104.9%

296.9%

200.8%

162.3%

106.0%

132.6%

152.9%

135.2%

2.5M - <5M

147.9%

172.4%

215.4%

198.4%

156.7%

111.8%

129.7%

553.2%

247.5%

277.8%

130.6%

163.4%

188.4%

166.6%

5M - <10M

331.2%

386.0%

317.9%

354.7%

1235.5%

298.9%

200.8%

1188.1%

554.2%

447.7%

292.4%

365.8%

520.8%

373.1%

10M+

756.3%

881.3%

725.8%

809.9%

5335.2%

682.4%

660.7%

1712.1%

1265.2%

1022.2%

667.7%

835.1%

7461.7%

851.7%

Face_Amount_Band

F

M

<10K

103.6%

145.2%

10K - <25K

100.9%

141.3%

25K - <50K

103.6%

150.3%

50K - <100K

98.9%

146.6%

100K - <250K

90.1%

127.2%

250K - <500K

97.3%

136.3%

500K - <1M

115.8%

162.3%

1M - <2.5M

104.9%

147.0%

2.5M - <5M

129.7%

212.9%

5M - <10M

200.8%

228.2%

10M+

660.7%

236.7%

AA_Grp

<10K

10K - <25K

25K - <50K

50K - <100K

100K - <250K

250K - <500K

500K - <1M

1M - <2.5M

2.5M - <5M

5M - <10M

10M+

0-17

105.9%

103.1%

105.9%

94.8%

94.1%

103.1%

124.7%

116.9%

132.6%

97.6%

343.8%

18-25

105.9%

103.1%

105.9%

94.8%

94.1%

103.1%

124.7%

116.9%

132.6%

97.6%

343.8%

26-30

105.9%

103.1%

105.9%

94.8%

94.1%

103.1%

124.7%

116.9%

132.6%

97.6%

343.8%

31-35

98.1%

95.5%

98.1%

87.9%

87.2%

95.5%

115.5%

108.3%

122.8%

90.4%

318.5%

36-40

83.5%

81.2%

83.5%

74.7%

74.1%

81.2%

98.2%

92.1%

104.4%

76.9%

270.8%

41-45

84.3%

82.0%

84.3%

75.4%

74.9%

82.0%

99.2%

102.2%

93.0%

77.6%

273.5%

46-50

103.0%

100.3%

103.0%

92.2%

91.5%

93.3%

119.8%

105.8%

128.9%

103.6%

334.4%

51-55

105.9%

103.1%

106.9%

100.0%

94.1%

95.9%

120.1%

116.9%

169.7%

97.6%

343.8%

56-60

103.6%

100.9%

103.6%

98.9%

90.1%

97.3%

115.8%

104.9%

129.7%

200.8%

660.7%

61-65

105.9%

103.1%

105.9%

94.8%

93.4%

103.1%

124.7%

106.0%

132.6%

51.6%

1539.9%

66-70

121.1%

117.9%

121.1%

108.5%

108.7%

118.3%

154.9%

137.7%

97.7%

110.6%

384.6%

71-75

132.6%

129.0%

132.6%

104.5%

117.8%

129.1%

181.2%

156.8%

165.9%

122.7%

430.3%

76-80

213.5%

207.8%

190.3%

169.2%

189.7%

241.3%

279.7%

235.5%

267.2%

196.7%

692.9%

81-85

179.3%

174.4%

179.3%

160.5%

169.7%

174.5%

211.0%

197.7%

224.3%

165.2%

581.8%

86-90

105.9%

103.1%

105.9%

94.8%

94.1%

103.1%

124.7%

116.9%

132.6%

97.6%

343.8%

91-95

105.9%

103.1%

105.9%

94.8%

94.1%

103.1%

124.7%

116.9%

132.6%

97.6%

343.8%

96-100

105.9%

103.1%

105.9%

94.8%

94.1%

103.1%

124.7%

116.9%

132.6%

97.6%

343.8%

106-110

105.9%

103.1%

105.9%

94.8%

94.1%

103.1%

124.7%

116.9%

132.6%

97.6%

343.8%

111-115

105.9%

103.1%

105.9%

94.8%

94.1%

103.1%

124.7%

116.9%

132.6%

97.6%

343.8%

Face_Amount_Band

5 yr anticipated

10 yr anticipated

15 yr anticipated

20 yr anticipated

25 yr anticipated

30 yr anticipated

Unknown

<10K

110.2%

192.9%

103.6%

125.9%

110.2%

110.2%

143.4%

10K - <25K

107.3%

187.7%

100.9%

122.5%

107.3%

107.3%

139.5%

25K - <50K

110.2%

192.9%

103.6%

125.9%

110.2%

110.2%

143.4%

50K - <100K

107.9%

188.9%

98.9%

123.3%

107.9%

107.9%

140.4%

100K - <250K

101.2%

175.0%

90.1%

115.6%

101.2%

101.2%

126.1%

250K - <500K

103.4%

181.0%

97.3%

118.2%

103.4%

103.4%

134.5%

500K - <1M

107.2%

187.5%

115.8%

122.4%

107.2%

107.2%

137.5%

1M - <2.5M

101.1%

176.9%

104.9%

115.5%

101.1%

101.1%

133.3%

2.5M - <5M

132.5%

231.9%

129.7%

151.4%

132.5%

132.5%

252.5%

5M - <10M

214.2%

491.7%

200.8%

244.6%

214.2%

214.2%

311.8%

10M+

299.6%

587.0%

660.7%

342.2%

299.6%

299.6%

389.7%

Sex

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

F

103.2%

130.4%

99.0%

104.8%

160.7%

93.1%

90.1%

214.8%

172.6%

139.4%

104.6%

126.2%

129.1%

116.2%

M

118.4%

151.5%

113.6%

120.3%

192.0%

97.5%

127.2%

246.6%

198.1%

172.8%

115.0%

144.9%

144.8%

155.5%

Dur_Grp

<10K

10K - <25K

25K - <50K

50K - <100K

100K - <250K

250K - <500K

500K - <1M

1M - <2.5M

2.5M - <5M

5M - <10M

10M+

6-10

99.5%

96.8%

99.5%

103.3%

89.4%

93.4%

111.2%

100.7%

124.5%

144.7%

634.4%

11-15

103.6%

100.9%

103.6%

98.9%

90.1%

97.3%

115.8%

104.9%

129.7%

200.8%

660.7%

16-20

141.3%

137.5%

141.3%

146.3%

122.4%

140.9%

157.9%

150.6%

176.7%

205.4%

1269.5%

21-30

87.0%

84.7%

87.0%

90.4%

82.4%

81.7%

102.7%

85.4%

108.8%

108.0%

554.8%

31-100

105.6%

102.8%

105.6%

109.7%

94.9%

99.1%

118.0%

106.9%

132.1%

153.6%

673.3%

AA_Grp

F

M

0-17

94.1%

130.9%

18-25

94.1%

130.9%

26-30

94.1%

130.9%

31-35

87.2%

121.3%

36-40

74.1%

103.1%

41-45

74.9%

104.1%

46-50

91.5%

127.3%

51-55

94.1%

128.3%

56-60

90.1%

127.2%

61-65

93.4%

129.9%

66-70

108.7%

151.1%

71-75

117.8%

169.3%

76-80

189.7%

263.8%

81-85

169.7%

236.0%

86-90

94.1%

130.9%

91-95

94.1%

130.9%

96-100

94.1%

130.9%

106-110

94.1%

130.9%

111-115

94.1%

130.9%

AA_Grp

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

0-17

107.7%

135.4%

103.4%

103.8%

160.9%

94.0%

94.1%

190.1%

180.2%

153.9%

109.2%

131.8%

115.6%

121.3%

18-25

107.7%

135.4%

103.4%

103.8%

160.9%

94.0%

94.1%

190.1%

180.2%

153.9%

109.2%

131.8%

115.6%

121.3%

26-30

107.7%

135.4%

103.4%

103.8%

160.9%

94.0%

94.1%

190.1%

180.2%

153.9%

109.2%

131.8%

115.6%

121.3%

31-35

99.8%

125.4%

95.8%

96.2%

149.0%

87.1%

87.2%

176.1%

167.0%

142.6%

101.2%

122.1%

107.1%

112.4%

36-40

84.9%

106.7%

81.4%

89.8%

126.7%

74.0%

74.1%

149.8%

142.0%

121.2%

86.0%

103.9%

91.1%

95.6%

41-45

85.7%

125.9%

82.2%

82.6%

128.0%

74.8%

74.9%

151.2%

143.4%

150.9%

86.9%

104.9%

92.0%

96.5%

46-50

104.8%

131.7%

100.5%

89.0%

156.4%

91.4%

91.5%

193.3%

175.3%

149.7%

106.2%

128.2%

112.4%

118.0%

51-55

107.7%

140.9%

97.8%

103.8%

160.9%

94.0%

94.1%

190.1%

180.2%

153.9%

109.2%

131.8%

115.6%

121.3%

56-60

103.2%

130.4%

99.0%

104.8%

160.7%

93.1%

90.1%

214.8%

172.6%

139.4%

104.6%

126.2%

129.1%

116.2%

61-65

106.9%

134.4%

102.6%

103.0%

192.9%

86.8%

93.4%

188.7%

182.6%

146.5%

108.4%

130.8%

114.7%

120.4%

66-70

124.4%

156.3%

119.4%

119.8%

163.3%

108.5%

108.7%

219.5%

208.1%

177.7%

126.1%

152.2%

133.5%

140.1%

71-75

134.8%

169.4%

87.4%

125.5%

201.3%

131.2%

117.8%

237.9%

225.5%

241.8%

136.7%

165.0%

149.0%

151.8%

76-80

174.2%

195.7%

170.1%

167.8%

222.6%

152.0%

189.7%

307.4%

291.4%

275.8%

176.6%

213.2%

187.0%

196.2%

81-85

194.3%

213.6%

186.4%

187.2%

295.3%

169.5%

169.7%

342.9%

325.0%

295.0%

196.9%

237.7%

208.5%

218.8%

86-90

107.7%

135.1%

103.4%

103.8%

160.9%

94.0%

94.1%

190.1%

180.2%

153.9%

109.2%

131.8%

115.6%

121.3%

91-95

107.7%

135.4%

103.4%

103.8%

160.9%

94.0%

94.1%

190.1%

180.2%

153.9%

109.2%

131.8%

115.6%

121.3%

96-100

107.7%

135.4%

103.4%

103.8%

160.9%

94.0%

94.1%

190.1%

180.2%

153.9%

109.2%

131.8%

115.6%

121.3%

106-110

107.7%

135.4%

103.4%

103.8%

160.9%

94.0%

94.1%

190.1%

180.2%

153.9%

109.2%

131.8%

115.6%

121.3%

111-115

107.7%

135.4%

103.4%

103.8%

160.9%

94.0%

94.1%

190.1%

180.2%

153.9%

109.2%

131.8%

115.6%

121.3%

SOA_Antp_Lvl_TP

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

5 yr anticipated

101.2%

131.1%

97.1%

105.1%

118.5%

104.7%

101.2%

210.7%

169.3%

98.9%

102.6%

123.9%

126.7%

114.0%

10 yr anticipated

150.6%

198.8%

144.5%

152.2%

212.3%

179.7%

175.0%

313.5%

270.6%

114.3%

160.6%

184.2%

188.4%

169.6%

15 yr anticipated

103.2%

130.4%

99.0%

104.8%

160.7%

93.1%

90.1%

214.8%

172.6%

139.4%

104.6%

126.2%

129.1%

116.2%

20 yr anticipated

115.6%

158.2%

111.0%

120.0%

135.3%

119.6%

115.6%

240.7%

193.4%

97.6%

117.2%

141.5%

144.7%

130.2%

25 yr anticipated

101.2%

131.1%

97.1%

105.1%

118.5%

104.7%

101.2%

210.7%

169.3%

98.9%

102.6%

123.9%

126.7%

114.0%

30 yr anticipated

101.2%

131.1%

97.1%

105.1%

118.5%

104.7%

101.2%

210.7%

169.3%

98.9%

102.6%

123.9%

126.7%

114.0%

Unknown

126.1%

161.8%

146.1%

151.3%

142.0%

130.4%

126.1%

357.4%

211.0%

116.0%

127.8%

154.3%

143.3%

142.0%

SOA_Antp_Lvl_TP

F

M

5 yr anticipated

101.2%

125.1%

10 yr anticipated

175.0%

216.3%

15 yr anticipated

90.1%

127.2%

20 yr anticipated

115.6%

155.0%

25 yr anticipated

101.2%

125.1%

30 yr anticipated

101.2%

125.1%

Unknown

126.1%

168.5%

AA_Grp

5 yr anticipated

10 yr anticipated

15 yr anticipated

20 yr anticipated

25 yr anticipated

30 yr anticipated

Unknown

0-17

105.7%

182.8%

94.1%

120.8%

105.7%

105.7%

127.0%

18-25

105.7%

182.8%

94.1%

120.8%

105.7%

105.7%

127.0%

26-30

105.7%

182.8%

94.1%

120.8%

105.7%

105.7%

127.0%

31-35

97.9%

169.3%

87.2%

111.9%

97.9%

97.9%

117.7%

36-40

83.3%

144.0%

74.1%

95.1%

83.3%

83.3%

100.1%

41-45

84.1%

125.6%

74.9%

96.0%

84.1%

84.1%

101.0%

46-50

102.8%

174.9%

91.5%

117.4%

102.8%

102.8%

123.5%

51-55

105.7%

182.8%

94.1%

117.6%

105.7%

105.7%

127.0%

56-60

101.2%

175.0%

90.1%

115.6%

101.2%

101.2%

126.1%

61-65

104.9%

184.5%

93.4%

128.0%

104.9%

104.9%

126.1%

66-70

108.2%

187.1%

108.7%

133.8%

108.2%

108.2%

130.1%

71-75

113.5%

223.3%

117.8%

140.4%

113.5%

113.5%

136.4%

76-80

200.3%

346.3%

189.7%

228.7%

200.3%

200.3%

298.9%

81-85

147.8%

255.5%

169.7%

145.1%

147.8%

147.8%

177.6%

86-90

105.7%

182.8%

94.1%

120.8%

105.7%

105.7%

127.0%

91-95

105.7%

182.8%

94.1%

120.8%

105.7%

105.7%

127.0%

96-100

105.7%

182.8%

94.1%

120.8%

105.7%

105.7%

127.0%

106-110

105.7%

182.8%

94.1%

120.8%

105.7%

105.7%

127.0%

111-115

105.7%

182.8%

94.1%

120.8%

105.7%

105.7%

127.0%

Dur_Grp

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

6-10

102.4%

129.4%

98.2%

103.9%

137.0%

92.4%

89.4%

137.0%

148.0%

134.0%

102.5%

125.3%

128.1%

115.3%

11-15

103.2%

130.4%

99.0%

104.8%

160.7%

93.1%

90.1%

214.8%

172.6%

139.4%

104.6%

126.2%

129.1%

116.2%

16-20

140.1%

177.2%

134.5%

131.7%

188.0%

126.4%

122.4%

187.6%

202.6%

183.5%

164.2%

171.5%

195.4%

157.8%

21-30

94.3%

119.2%

90.5%

95.7%

126.2%

85.1%

82.4%

126.2%

136.3%

122.4%

97.7%

115.4%

120.1%

110.3%

31-100

108.6%

137.4%

104.3%

110.3%

145.4%

98.0%

94.9%

145.4%

157.1%

142.2%

108.8%

132.9%

136.0%

122.3%

Dur_Grp

F

M

6-10

89.4%

126.2%

11-15

90.1%

127.2%

16-20

122.4%

177.9%

21-30

82.4%

124.9%

31-100

94.9%

134.3%

Dur_Grp

5 yr anticipated

10 yr anticipated

15 yr anticipated

20 yr anticipated

25 yr anticipated

30 yr anticipated

Unknown

6-10

100.4%

130.9%

89.4%

114.7%

100.4%

100.4%

101.2%

11-15

101.2%

175.0%

90.1%

115.6%

101.2%

101.2%

126.1%

16-20

104.7%

136.5%

122.4%

119.6%

104.7%

104.7%

112.8%

21-30

105.8%

137.9%

82.4%

122.9%

105.8%

105.8%

113.9%

31-100

100.4%

130.9%

94.9%

114.7%

100.4%

100.4%

101.2%

AA_Grp

6-10

11-15

16-20

21-30

31-100

0-17

91.4%

94.1%

126.5%

84.2%

97.0%

18-25

91.4%

94.1%

126.5%

84.2%

97.0%

26-30

91.4%

94.1%

126.5%

84.2%

97.0%

31-35

84.6%

87.2%

117.2%

78.0%

89.8%

36-40

87.7%

74.1%

121.5%

80.8%

93.1%

41-45

79.6%

74.9%

110.3%

73.3%

84.5%

46-50

88.8%

91.5%

114.5%

81.8%

94.3%

51-55

91.4%

94.1%

126.5%

81.6%

97.0%

56-60

89.4%

90.1%

122.4%

82.4%

94.9%

61-65

90.7%

93.4%

123.7%

90.6%

96.2%

66-70

105.5%

108.7%

155.0%

114.2%

112.0%

71-75

114.3%

117.8%

168.6%

113.5%

121.3%

76-80

161.2%

189.7%

223.2%

148.4%

171.1%

81-85

160.1%

169.7%

226.9%

147.4%

169.9%

86-90

91.4%

94.1%

126.5%

84.2%

97.0%

91-95

91.4%

94.1%

126.5%

84.2%

97.0%

96-100

91.4%

94.1%

126.5%

84.2%

97.0%

106-110

91.4%

94.1%

126.5%

84.2%

97.0%

111-115

91.4%

94.1%

126.5%

84.2%

97.0%

7.2.6.2 Goodness-of-Fit

Next comes checking goodness-of-fit, on both a univariate and bivariate basis. Tables are provided which show the ratio of the actual to model predicted deaths and show the associated death counts.

Code
dat.term.plt.grid %>%
  right_join(dat.term.plt) %>%
  mutate(ExpDth_Amt_GLMNet=ExpDth_Amt_VBT2015*Factor) %>%
  mutate(Factor=NULL) ->
  dat.term.plt
7.2.6.2.1 Univariate Fit Checks

Sex

Death Count

Actual-to-Model

F

17,571

100.3%

M

24,955

99.9%

Face_Amount_Band

Death Count

Actual-to-Model

<10K

903

86.4%

10K - <25K

6,652

95.4%

25K - <50K

7,682

101.7%

50K - <100K

8,892

100.6%

100K - <250K

12,915

99.8%

250K - <500K

3,439

99.6%

500K - <1M

1,415

99.6%

1M - <2.5M

563

99.7%

2.5M - <5M

45

103.0%

5M - <10M

17

101.4%

10M+

3

111.6%

SOA_Antp_Lvl_TP

Death Count

Actual-to-Model

5 yr anticipated

9,885

98.4%

10 yr anticipated

11,750

100.2%

15 yr anticipated

5,440

100.5%

20 yr anticipated

10,861

100.8%

25 yr anticipated

0

0.0%

30 yr anticipated

0

0.0%

Unknown

4,590

100.6%

UW

Death Count

Actual-to-Model

NS/1/2

3,872

95.9%

NS/2/2

14,000

100.6%

NS/1/3

1,016

98.8%

NS/2/3

1,263

101.1%

NS/3/3

1,685

100.8%

NS/1/4

789

100.8%

NS/2/4

905

99.1%

NS/3/4

452

102.8%

NS/4/4

536

104.3%

NS/U/U

7,755

100.3%

S/1/2

1,354

98.0%

S/2/2

664

104.1%

S/U/U

5,935

101.1%

U/U/U

2,300

104.8%

AA_Grp

Death Count

Actual-to-Model

0-17

1

293.9%

18-25

29

67.1%

26-30

90

89.7%

31-35

208

81.6%

36-40

416

94.7%

41-45

1,068

97.8%

46-50

2,195

99.1%

51-55

4,236

99.9%

56-60

5,910

100.3%

61-65

6,974

100.2%

66-70

6,547

100.5%

71-75

4,658

100.8%

76-80

4,751

101.3%

81-85

3,582

102.5%

86-90

1,487

100.3%

91-95

359

78.4%

96-100

15

64.4%

106-110

0

0.0%

111-115

0

0.0%

Dur_Grp

Death Count

Actual-to-Model

6-10

1,273

95.0%

11-15

9,129

100.2%

16-20

10,770

100.2%

21-30

18,917

100.2%

31-100

2,437

102.0%

7.2.6.2.2 Bivariate Fit Checks

Face_Amount_Band

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

<10K

629

88.1%

274

83.9%

10K - <25K

3,926

94.9%

2,726

95.9%

25K - <50K

3,557

99.6%

4,125

103.2%

50K - <100K

3,731

99.4%

5,161

101.3%

100K - <250K

4,504

98.7%

8,411

100.4%

250K - <500K

888

98.2%

2,551

100.1%

500K - <1M

253

99.3%

1,162

99.7%

1M - <2.5M

72

100.5%

491

99.6%

2.5M - <5M

4

100.2%

41

103.3%

5M - <10M

5

127.8%

12

93.7%

10M+

2

210.2%

1

70.4%

SOA_Antp_Lvl_TP

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

5 yr anticipated

3,880

99.8%

6,005

97.8%

10 yr anticipated

4,537

100.8%

7,213

100.0%

15 yr anticipated

2,137

100.0%

3,303

100.7%

20 yr anticipated

5,461

100.2%

5,400

101.0%

25 yr anticipated

0

0.0%

0

0.0%

30 yr anticipated

0

0.0%

0

0.0%

Unknown

1,556

99.7%

3,034

100.8%

UW

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

NS/1/2

1,447

95.1%

2,425

96.3%

NS/2/2

7,291

100.4%

6,709

100.7%

NS/1/3

386

93.3%

630

101.2%

NS/2/3

417

105.4%

846

99.8%

NS/3/3

564

99.8%

1,121

101.1%

NS/1/4

275

109.0%

514

97.9%

NS/2/4

223

83.7%

682

102.3%

NS/3/4

125

108.0%

327

101.7%

NS/4/4

162

109.3%

374

102.9%

NS/U/U

2,770

99.5%

4,985

100.6%

S/1/2

456

99.4%

898

97.4%

S/2/2

212

119.8%

452

99.5%

S/U/U

2,418

106.6%

3,517

98.5%

U/U/U

825

99.2%

1,475

106.4%

AA_Grp

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

0-17

0

0.0%

1

442.9%

18-25

7

67.0%

22

67.1%

26-30

24

69.9%

66

98.5%

31-35

72

70.0%

136

88.7%

36-40

169

89.8%

247

97.6%

41-45

440

98.9%

628

97.2%

46-50

842

95.1%

1,353

100.8%

51-55

1,716

101.3%

2,520

99.3%

56-60

2,332

99.5%

3,578

100.6%

61-65

2,701

100.3%

4,273

100.1%

66-70

2,559

101.8%

3,988

100.1%

71-75

1,944

100.1%

2,714

101.0%

76-80

2,062

102.9%

2,689

100.7%

81-85

1,672

108.6%

1,910

99.6%

86-90

804

119.6%

683

89.8%

91-95

216

93.8%

143

72.8%

96-100

11

87.4%

4

38.5%

106-110

0

0.0%

111-115

0

0.0%

Dur_Grp

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

6-10

546

106.8%

727

90.1%

11-15

3,391

100.0%

5,738

100.3%

16-20

4,224

100.0%

6,546

100.3%

21-30

8,568

99.8%

10,349

100.4%

31-100

842

95.3%

1,595

104.1%

Face_Amount_Band

SOA_Antp_Lvl_TP: 5 yr anticipated

SOA_Antp_Lvl_TP: 10 yr anticipated

SOA_Antp_Lvl_TP: 15 yr anticipated

SOA_Antp_Lvl_TP: 20 yr anticipated

SOA_Antp_Lvl_TP: 25 yr anticipated

SOA_Antp_Lvl_TP: 30 yr anticipated

SOA_Antp_Lvl_TP: Unknown

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

<10K

48

87.4%

207

92.7%

65

91.3%

559

83.7%

0

0.0%

0

0.0%

24

84.6%

10K - <25K

331

107.5%

1,734

94.3%

533

88.6%

3,793

94.3%

0

0.0%

0

0.0%

261

115.1%

25K - <50K

2,537

111.1%

1,194

96.2%

734

90.7%

2,149

97.3%

0

0.0%

0

0.0%

1,068

103.9%

50K - <100K

2,792

105.2%

1,851

99.7%

1,096

93.9%

2,007

101.0%

0

0.0%

1,146

97.2%

100K - <250K

3,183

101.7%

4,223

99.3%

1,993

98.5%

2,040

101.0%

0

0.0%

0

0.0%

1,476

97.9%

250K - <500K

694

98.2%

1,504

99.5%

616

99.7%

221

102.6%

0

0.0%

404

100.9%

500K - <1M

212

96.1%

706

99.8%

289

102.7%

70

105.7%

138

95.4%

1M - <2.5M

82

86.3%

294

100.7%

105

103.2%

20

110.9%

62

105.5%

2.5M - <5M

5

68.1%

23

104.2%

7

119.7%

2

97.2%

8

124.4%

5M - <10M

1

58.1%

12

108.0%

1

58.4%

0

0.0%

3

136.0%

10M+

0

0.0%

2

118.9%

1

149.2%

0

0.0%

0

0.0%

UW

Face_Amount_Band: <10K

Face_Amount_Band: 10K - <25K

Face_Amount_Band: 25K - <50K

Face_Amount_Band: 50K - <100K

Face_Amount_Band: 100K - <250K

Face_Amount_Band: 250K - <500K

Face_Amount_Band: 500K - <1M

Face_Amount_Band: 1M - <2

Face_Amount_Band: 2

Face_Amount_Band: 5M - <10M

Face_Amount_Band: 10M+

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

5M

5M - <5M

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

NS/1/2

16

67.4%

144

79.9%

350

86.9%

611

100.0%

1,804

100.6%

676

95.3%

197

91.0%

69

95.7%

4

100.4%

1

66.7%

0

0.0%

NS/2/2

600

83.3%

4,492

93.0%

2,822

102.4%

2,641

103.2%

2,828

101.3%

489

102.9%

108

105.6%

19

84.0%

1

64.7%

0

0.0%

0

0.0%

NS/1/3

0

0.0%

0

0.0%

19

60.5%

72

78.9%

453

95.1%

242

97.9%

161

104.4%

60

96.0%

8

121.9%

1

69.2%

0

0.0%

NS/2/3

0

0.0%

1

153.3%

31

64.4%

146

108.3%

551

95.0%

302

101.4%

157

100.0%

66

104.8%

7

124.0%

2

118.5%

0

0.0%

NS/3/3

3

58.0%

26

100.4%

117

75.4%

233

88.3%

752

96.1%

314

99.7%

142

95.1%

87

104.3%

2

51.9%

7

122.2%

2

141.3%

NS/1/4

17

53.8%

80

72.3%

336

102.3%

166

101.6%

123

107.1%

65

106.6%

1

47.5%

1

80.5%

0

0.0%

NS/2/4

3

862.1%

20

48.6%

148

77.3%

407

93.4%

175

106.4%

105

107.0%

42

100.1%

5

129.9%

0

0.0%

0

0.0%

NS/3/4

8

40.0%

48

51.7%

202

86.4%

95

92.3%

60

98.4%

30

111.5%

6

140.5%

3

153.1%

0

0.0%

NS/4/4

1

1006.2%

22

50.6%

101

81.4%

245

102.9%

96

115.4%

52

116.7%

18

103.9%

1

88.7%

0

0.0%

0

0.0%

NS/U/U

21

82.2%

341

124.4%

1,598

109.4%

2,216

103.5%

2,822

100.6%

487

99.1%

190

96.7%

69

96.6%

10

118.3%

1

78.5%

0

0.0%

S/1/2

1

83.3%

1

424.2%

97

101.7%

302

106.3%

716

105.0%

160

97.2%

55

94.7%

22

95.4%

0

0.0%

0

0.0%

0

0.0%

S/2/2

0

0.0%

0

0.0%

44

110.6%

133

94.0%

355

110.6%

95

117.0%

29

106.1%

8

81.9%

0

0.0%

0

0.0%

0

0.0%

S/U/U

157

108.9%

962

98.4%

1,796

104.6%

1,621

102.3%

1,235

97.5%

126

90.7%

30

83.3%

6

67.3%

0

0.0%

1

723.3%

1

361.5%

U/U/U

105

82.0%

681

96.8%

741

104.8%

540

108.4%

209

107.2%

16

97.1%

6

134.0%

2

104.0%

0

0.0%

0

0.0%

0

0.0%

AA_Grp

Face_Amount_Band: <10K

Face_Amount_Band: 10K - <25K

Face_Amount_Band: 25K - <50K

Face_Amount_Band: 50K - <100K

Face_Amount_Band: 100K - <250K

Face_Amount_Band: 250K - <500K

Face_Amount_Band: 500K - <1M

Face_Amount_Band: 1M - <2

Face_Amount_Band: 2

Face_Amount_Band: 10M+

Face_Amount_Band: 5M - <10M

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

5M

5M - <5M

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

0-17

0

0.0%

0

0.0%

0

0.0%

1

773.0%

0

0.0%

0

0.0%

0

0.0%

18-25

0

0.0%

0

0.0%

3

162.3%

23

82.9%

3

59.9%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

26-30

0

0.0%

2

239.3%

17

78.8%

51

98.4%

17

111.0%

3

107.0%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

31-35

2

2106.3%

2

174.8%

71

127.8%

69

131.9%

49

84.9%

11

67.7%

3

58.5%

1

53.3%

0

0.0%

0

0.0%

0

0.0%

36-40

1

0.0%

9

136.1%

76

109.7%

101

140.0%

135

86.7%

58

86.9%

28

110.0%

7

72.0%

1

207.2%

0

0.0%

0

0.0%

41-45

3

91.1%

50

240.6%

140

130.4%

206

123.5%

415

95.0%

164

99.3%

60

94.0%

30

112.0%

0

0.0%

0

0.0%

0

0.0%

46-50

6

181.1%

80

147.6%

238

136.6%

399

112.1%

958

99.6%

318

95.9%

138

95.3%

52

94.0%

4

112.9%

0

0.0%

2

163.4%

51-55

21

158.9%

186

183.6%

507

137.7%

870

109.0%

1,802

99.1%

531

97.4%

218

97.0%

88

98.1%

11

116.5%

0

0.0%

2

96.1%

56-60

25

55.1%

260

114.0%

791

105.1%

1,463

105.4%

2,341

98.7%

659

98.0%

257

97.4%

100

96.7%

7

101.5%

1

166.0%

6

124.2%

61-65

50

68.5%

466

109.8%

1,309

110.6%

1,672

99.1%

2,484

98.8%

649

101.5%

247

98.2%

88

96.1%

7

113.3%

2

142.2%

0

0.0%

66-70

72

87.1%

557

103.4%

1,398

105.8%

1,646

98.8%

2,096

101.5%

490

102.7%

209

103.5%

76

104.2%

2

57.7%

0

0.0%

1

58.3%

71-75

86

80.6%

768

93.7%

1,101

95.2%

995

92.8%

1,245

100.5%

260

96.6%

135

105.9%

59

105.8%

7

111.9%

0

0.0%

2

148.3%

76-80

163

69.3%

1,438

84.9%

1,098

88.1%

843

91.9%

876

101.7%

204

107.7%

85

109.7%

38

107.4%

4

123.4%

0

0.0%

2

106.0%

81-85

226

98.9%

1,675

97.5%

717

87.5%

436

94.1%

399

109.2%

76

117.5%

32

112.0%

18

98.4%

1

108.6%

0

0.0%

2

108.6%

86-90

184

91.5%

901

88.3%

190

84.8%

104

100.7%

84

138.3%

16

135.2%

2

63.1%

6

122.5%

0

0.0%

0

0.0%

91-95

60

82.8%

248

83.8%

25

87.0%

13

122.9%

11

143.2%

0

0.0%

1

95.0%

0

0.0%

1

61.0%

96-100

4

75.9%

10

85.5%

1

41.4%

0

0.0%

106-110

0

0.0%

111-115

0

0.0%

Face_Amount_Band

Dur_Grp: 6-10

Dur_Grp: 11-15

Dur_Grp: 16-20

Dur_Grp: 21-30

Dur_Grp: 31-100

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

<10K

0

0.0%

65

90.0%

87

92.6%

647

85.3%

104

86.0%

10K - <25K

44

221.5%

617

87.9%

655

88.6%

4,731

96.0%

605

102.5%

25K - <50K

100

180.0%

1,052

103.0%

1,761

98.3%

3,985

101.5%

784

102.7%

50K - <100K

426

137.1%

1,491

95.3%

2,316

97.2%

4,136

101.6%

523

103.2%

100K - <250K

420

96.5%

3,504

99.1%

3,977

99.2%

4,635

100.7%

379

107.2%

250K - <500K

149

83.8%

1,393

99.9%

1,293

101.1%

570

101.6%

34

88.3%

500K - <1M

81

95.2%

667

99.7%

487

99.2%

175

104.1%

5

69.8%

1M - <2.5M

46

92.0%

299

100.7%

180

101.8%

36

91.7%

2

103.0%

2.5M - <5M

6

109.2%

26

104.5%

10

99.6%

2

83.2%

1

105.5%

5M - <10M

1

70.9%

13

107.2%

3

123.0%

0

0.0%

10M+

0

0.0%

2

112.1%

1

131.5%

0

0.0%

UW

SOA_Antp_Lvl_TP: 5 yr anticipated

SOA_Antp_Lvl_TP: 10 yr anticipated

SOA_Antp_Lvl_TP: 15 yr anticipated

SOA_Antp_Lvl_TP: 20 yr anticipated

SOA_Antp_Lvl_TP: Unknown

SOA_Antp_Lvl_TP: 30 yr anticipated

SOA_Antp_Lvl_TP: 25 yr anticipated

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

NS/1/2

881

92.7%

1,065

92.9%

643

102.9%

783

97.8%

500

98.7%

NS/2/2

1,415

103.0%

3,080

102.0%

1,091

95.7%

7,878

101.4%

536

94.7%

0

0.0%

NS/1/3

117

89.6%

441

99.9%

321

100.1%

29

78.0%

108

112.8%

NS/2/3

148

107.8%

569

97.9%

360

95.6%

68

118.6%

118

112.1%

NS/3/3

235

96.2%

985

101.7%

288

105.7%

20

139.8%

157

88.5%

NS/1/4

19

97.6%

465

102.6%

225

94.0%

2

201.5%

78

109.5%

NS/2/4

15

40.3%

594

102.7%

220

92.8%

1

178.0%

75

99.7%

NS/3/4

5

44.9%

312

99.8%

81

102.5%

0

0.0%

54

126.2%

NS/4/4

16

95.1%

343

105.1%

124

100.5%

0

0.0%

53

116.3%

NS/U/U

3,674

103.8%

1,551

98.3%

575

104.8%

456

93.7%

1,499

97.5%

0

0.0%

0

0.0%

S/1/2

337

79.1%

543

104.5%

249

98.9%

65

160.3%

160

109.1%

S/2/2

164

91.5%

288

108.4%

125

106.6%

15

132.1%

72

112.2%

S/U/U

2,126

103.0%

1,251

101.0%

291

110.4%

1,470

101.6%

797

91.6%

0

0.0%

0

0.0%

U/U/U

733

104.4%

263

94.2%

847

112.7%

74

93.1%

383

105.8%

0

0.0%

0

0.0%

AA_Grp

SOA_Antp_Lvl_TP: 5 yr anticipated

SOA_Antp_Lvl_TP: 10 yr anticipated

SOA_Antp_Lvl_TP: 15 yr anticipated

SOA_Antp_Lvl_TP: Unknown

SOA_Antp_Lvl_TP: 20 yr anticipated

SOA_Antp_Lvl_TP: 25 yr anticipated

SOA_Antp_Lvl_TP: 30 yr anticipated

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

0-17

0

0.0%

1

399.9%

0

0.0%

0

0.0%

18-25

17

66.1%

2

51.3%

0

0.0%

9

70.5%

1

3643.5%

0

0.0%

26-30

47

90.1%

9

66.1%

1

25.9%

33

107.1%

0

0.0%

0

0.0%

31-35

124

92.1%

40

72.8%

3

68.0%

41

72.6%

0

0.0%

0

0.0%

36-40

193

103.5%

121

90.6%

4

37.5%

89

90.3%

9

120.4%

0

0.0%

41-45

407

104.5%

310

94.5%

38

62.7%

201

98.3%

112

122.4%

0

0.0%

46-50

782

104.1%

602

97.9%

159

90.8%

367

94.8%

285

109.2%

0

0.0%

51-55

1,374

100.2%

1,164

101.1%

423

96.9%

684

102.0%

591

92.5%

0

0.0%

0

0.0%

56-60

1,777

100.1%

1,548

100.3%

707

98.0%

798

103.4%

1,080

100.1%

0

0.0%

0

0.0%

61-65

2,014

96.2%

1,829

101.1%

842

98.8%

927

103.4%

1,362

103.8%

0

0.0%

0

0.0%

66-70

1,728

95.6%

1,677

101.2%

1,040

102.8%

655

98.1%

1,447

104.4%

0

0.0%

0

0.0%

71-75

709

90.0%

1,448

102.0%

914

103.6%

347

94.3%

1,240

106.7%

0

0.0%

0

0.0%

76-80

416

85.4%

1,441

100.0%

910

103.8%

273

111.4%

1,711

103.3%

0

0.0%

81-85

237

103.9%

1,171

102.8%

334

111.9%

127

113.1%

1,713

89.6%

0

0.0%

86-90

56

139.1%

376

97.6%

58

143.1%

33

109.8%

964

84.4%

0

0.0%

91-95

4

269.7%

11

114.9%

7

85.0%

5

11.6%

332

89.7%

96-100

0

0.0%

1

7.7%

14

72.8%

106-110

0

0.0%

111-115

0

0.0%

SOA_Antp_Lvl_TP

Dur_Grp: 6-10

Dur_Grp: 11-15

Dur_Grp: 16-20

Dur_Grp: 21-30

Dur_Grp: 31-100

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

5 yr anticipated

949

97.9%

1,577

97.2%

3,033

98.8%

3,339

99.8%

987

95.6%

10 yr anticipated

5,922

100.3%

3,018

99.8%

2,496

100.1%

314

104.7%

15 yr anticipated

3,407

100.7%

1,462

96.9%

571

122.5%

20 yr anticipated

10,571

100.7%

290

102.1%

25 yr anticipated

0

0.0%

0

0.0%

30 yr anticipated

0

0.0%

Unknown

324

84.3%

1,630

101.5%

1,312

102.6%

1,049

105.4%

275

98.0%

AA_Grp

UW: NS/U/U

UW: S/U/U

UW: U/U/U

UW: NS/1/2

UW: NS/2/2

UW: NS/1/3

UW: NS/2/3

UW: NS/3/3

UW: NS/1/4

UW: NS/4/4

UW: S/1/2

UW: S/2/2

UW: NS/2/4

UW: NS/3/4

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

0-17

1

299.9%

0

0.0%

0

0.0%

18-25

5

81.6%

1

50.2%

22

73.8%

0

0.0%

1

606.7%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

26-30

24

113.5%

10

103.3%

42

87.1%

2

287.7%

2

51.4%

3

58.6%

1

41.0%

1

26.4%

1

69.4%

0

0.0%

2

65.6%

2

428.1%

0

0.0%

0

0.0%

31-35

62

101.4%

32

73.6%

47

115.7%

9

74.4%

11

63.5%

11

73.6%

8

101.2%

9

62.2%

2

61.7%

0

0.0%

9

84.4%

6

118.6%

1

15.8%

1

114.8%

36-40

99

88.0%

68

73.4%

14

103.3%

46

91.5%

71

121.5%

28

95.7%

21

144.4%

28

94.8%

7

32.5%

2

42.2%

15

73.8%

8

72.3%

5

96.0%

4

86.4%

41-45

269

116.0%

159

87.8%

31

221.5%

114

103.8%

256

117.5%

43

104.9%

43

99.8%

62

92.4%

15

71.7%

5

65.3%

41

89.8%

18

60.6%

9

48.4%

3

19.8%

46-50

529

96.3%

354

91.9%

51

173.1%

290

100.9%

479

99.5%

81

99.3%

63

85.7%

109

106.7%

37

98.2%

24

150.6%

83

84.1%

56

85.2%

20

104.3%

19

139.3%

51-55

1,036

97.2%

748

101.0%

168

153.8%

475

94.1%

940

103.7%

113

92.5%

136

106.6%

136

93.9%

75

95.8%

34

117.3%

186

106.2%

96

119.4%

64

108.7%

29

92.2%

56-60

1,366

97.7%

1,043

104.9%

347

117.7%

602

89.5%

1,369

102.9%

141

105.2%

168

105.7%

221

105.5%

113

109.2%

47

93.3%

236

91.8%

116

99.0%

84

89.6%

57

116.6%

61-65

1,536

97.9%

1,179

100.0%

571

99.0%

643

97.8%

1,661

102.4%

173

97.2%

195

101.7%

262

104.8%

122

89.3%

68

122.1%

257

92.6%

136

108.6%

112

104.0%

59

105.6%

66-70

1,320

100.3%

994

101.4%

500

99.8%

576

100.4%

1,692

100.1%

142

92.0%

206

100.9%

262

92.8%

125

106.5%

90

111.7%

281

111.6%

117

115.9%

158

92.3%

84

106.9%

71-75

673

106.0%

594

115.4%

222

93.2%

389

94.9%

1,613

102.5%

97

77.0%

181

90.9%

237

105.1%

103

112.3%

92

81.1%

141

106.5%

62

109.8%

169

106.1%

85

95.4%

76-80

460

110.0%

402

106.9%

165

95.7%

329

102.2%

2,322

94.3%

116

115.2%

152

86.4%

202

87.4%

115

102.7%

115

94.9%

77

116.8%

39

134.4%

184

111.4%

73

104.5%

81-85

257

118.3%

257

122.4%

97

84.3%

280

91.5%

2,163

91.7%

58

133.3%

73

121.5%

129

113.8%

66

132.9%

51

111.0%

24

97.2%

5

38.5%

87

73.7%

35

59.7%

86-90

93

123.9%

80

100.0%

20

160.0%

95

81.1%

1,114

82.4%

9

176.6%

15

128.3%

26

158.3%

8

44.7%

8

145.2%

2

166.5%

2

284.2%

12

58.4%

3

161.7%

91-95

24

68.5%

14

123.7%

2

80.7%

22

108.6%

293

85.1%

1

74.1%

1

893.4%

1

44.3%

0

0.0%

0

0.0%

1

137.2%

0

0.0%

96-100

1

44.7%

1

27.2%

0

0.0%

13

82.7%

106-110

0

0.0%

111-115

0

0.0%

UW

Dur_Grp: 6-10

Dur_Grp: 11-15

Dur_Grp: 16-20

Dur_Grp: 21-30

Dur_Grp: 31-100

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

NS/1/2

24

87.1%

558

84.3%

1,377

98.8%

1,892

99.3%

21

84.4%

NS/2/2

104

150.3%

2,054

101.1%

2,578

100.0%

9,110

99.4%

154

132.1%

NS/1/3

88

98.6%

413

100.3%

445

99.1%

70

85.3%

0

0.0%

NS/2/3

119

112.7%

490

100.6%

528

96.7%

126

102.0%

0

0.0%

NS/3/3

218

95.1%

804

101.6%

454

103.3%

204

93.5%

5

116.5%

NS/1/4

7

34.1%

488

101.2%

289

100.8%

5

291.8%

NS/2/4

8

46.1%

621

100.4%

274

97.5%

2

253.6%

NS/3/4

4

54.0%

354

104.0%

94

98.3%

0

0.0%

NS/4/4

5

69.7%

380

105.2%

151

104.8%

0

0.0%

NS/U/U

268

140.3%

1,182

101.9%

2,296

99.6%

3,562

98.8%

447

98.8%

S/1/2

118

58.1%

473

104.6%

508

106.3%

251

117.5%

4

219.3%

S/2/2

65

88.0%

296

106.5%

230

105.2%

72

119.9%

1

267.7%

S/U/U

213

109.8%

963

96.7%

1,498

103.7%

2,991

102.8%

270

90.7%

U/U/U

32

80.3%

53

72.9%

48

101.0%

632

114.2%

1,535

103.1%

AA_Grp

Dur_Grp: 6-10

Dur_Grp: 11-15

Dur_Grp: 16-20

Dur_Grp: 21-30

Dur_Grp: 31-100

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

0-17

0

0.0%

1

426.1%

0

0.0%

18-25

25

66.0%

1

34.4%

2

106.9%

1

302.7%

26-30

39

87.2%

49

96.2%

2

29.3%

0

0.0%

0

0.0%

31-35

38

68.3%

124

86.1%

45

111.4%

1

14.0%

0

0.0%

36-40

85

102.7%

197

89.6%

103

100.9%

30

97.4%

1

64.6%

41-45

113

100.3%

426

96.1%

300

96.1%

223

113.8%

6

259.6%

46-50

152

88.5%

697

100.6%

690

96.8%

627

105.4%

29

126.3%

51-55

193

91.9%

1,082

101.1%

1,259

100.7%

1,550

97.4%

152

127.5%

56-60

184

117.3%

1,199

98.9%

1,598

98.7%

2,523

100.2%

406

117.1%

61-65

170

94.5%

1,364

101.0%

1,736

98.7%

3,014

101.6%

690

101.0%

66-70

129

84.7%

1,207

100.3%

1,694

101.6%

2,870

102.1%

647

94.5%

71-75

67

68.4%

1,072

100.5%

1,267

102.3%

2,030

103.4%

222

98.7%

76-80

55

84.8%

1,064

103.3%

1,242

101.6%

2,234

99.0%

156

110.2%

81-85

23

141.3%

583

107.1%

649

106.4%

2,233

94.7%

94

67.4%

86-90

63

118.3%

179

112.0%

1,222

89.8%

23

137.1%

91-95

0

0.0%

4

77.0%

346

71.7%

9

96.7%

96-100

13

64.6%

2

63.5%

106-110

0

0.0%

111-115

0

0.0%

8 Perm Modeling by Amount

8.1 Perm Model, Unknown Smoker

Code
dat.perm.unk <- readRDS("dat.perm.rds")
dat.perm.unk <- dat.perm.unk[Policies_Exposed > 0]
dat.perm.unk <- dat.perm.unk[UW == "U/U/U" &
                               Insurance_Plan == "Perm"]

bUseAllInteractions <- FALSE

AA.brks <- c(-1,17,
             seq(25,120,5))
AA.lbls <- paste0(
  AA.brks[1:(length(AA.brks)-1)]+1,
  "-",
  AA.brks[2:length(AA.brks)]
)

Dur.brks <- c(0,1,2,3,5,10,15,20,30,120)
Dur.lbls <- paste0(
  Dur.brks[1:(length(Dur.brks)-1)]+1,
  "-",
  Dur.brks[2:length(Dur.brks)]
)

dat.perm.unk[,`:=`(
  AA_Grp=cut(Attained_Age,
              breaks=AA.brks,
              labels=AA.lbls),
  Dur_Grp=cut(Duration,
               breaks=Dur.brks,
               labels=Dur.lbls)
)]



pred.cols <- names(dat.perm.unk)[c(1,5,16:17)]

factor.cols <- pred.cols

dat.perm.unk[,
      (factor.cols):=lapply(.SD,factor),
      .SDcols=factor.cols]

set.seed(traintest.seed)
dat.perm.unk %>%
  mutate(
    IsTraining = (runif(nrow(.)) < training.fraction),
    Noise = rnorm(nrow(.)),
    .before = 1
  ) %>% 
  mutate(Face_Amount_Band=fct_relabel(
    Face_Amount_Band,
    #function(.) sub(":"," -",.,fixed=T)
    function(.) fa.remap[Face_Amount_Band.Old==.,Face_Amount_Band.New]
    ),
    AE_Count=Death_Count/ExpDth_Cnt_VBT2015,
    AE_Amount=Death_Claim_Amount/ExpDth_Amt_VBT2015
  ) -> 
  dat.perm.unk

gbm.pred.cols <- c("Noise",pred.cols)

Here is a data preview.

Code
dat.perm.unk %>%
  head(10) %>%
  flextable()

IsTraining

Noise

Sex

Smoker_Status

Attained_Age

Duration

Face_Amount_Band

Insurance_Plan

Number_of_Pfd_Classes

Preferred_Class

Death_Count

Policies_Exposed

ExpDth_Cnt_VBT2015

Death_Claim_Amount

Amount_Exposed

ExpDth_Amt_VBT2015

UW

AA_Grp

Dur_Grp

AE_Count

AE_Amount

TRUE

-1.31531994

M

U

72

56

250K - <500K

Perm

U

U

0

5.465753

0.1014444

0

1,745,572

32,397.82

U/U/U

71-75

31-120

0.0000000

0.0000000

TRUE

-0.65764994

M

U

73

57

10K - <25K

Perm

U

U

151

7,161.337555

150.0379591

1,979,552

95,811,011

2,007,923.42

U/U/U

71-75

31-120

1.0064120

0.9858703

TRUE

0.78905215

M

U

73

57

25K - <50K

Perm

U

U

26

1,271.731550

26.6477580

938,930

42,122,230

882,099.08

U/U/U

71-75

31-120

0.9756918

1.0644269

TRUE

-1.52205566

M

U

73

57

100K - <250K

Perm

U

U

2

75.895890

1.5842799

295,327

10,165,449

212,139.10

U/U/U

71-75

31-120

1.2624032

1.3921385

TRUE

0.07196495

M

U

74

58

50K - <100K

Perm

U

U

4

328.650822

7.7000963

270,364

21,161,507

495,822.79

U/U/U

71-75

31-120

0.5194740

0.5452835

TRUE

-0.18478717

M

U

74

58

250K - <500K

Perm

U

U

0

6.336987

0.1482855

0

2,052,935

48,038.67

U/U/U

71-75

31-120

0.0000000

0.0000000

FALSE

0.47450347

M

U

75

59

<10K

Perm

U

U

469

19,436.460915

511.1919127

1,892,560

81,348,082

2,137,055.46

U/U/U

71-75

31-120

0.9174637

0.8855924

TRUE

-1.04162923

M

U

75

59

250K - <500K

Perm

U

U

0

6.361337

0.1667943

0

2,021,135

52,994.16

U/U/U

71-75

31-120

0.0000000

0.0000000

TRUE

0.40207422

M

U

76

60

<10K

Perm

U

U

523

18,052.611237

529.5150148

2,156,603

74,124,703

2,171,516.43

U/U/U

76-80

31-120

0.9876963

0.9931323

TRUE

1.01522821

M

U

76

60

10K - <25K

Perm

U

U

97

3,605.502399

105.6160386

1,358,119

49,470,772

1,448,978.11

U/U/U

76-80

31-120

0.9184211

0.9372944

8.1.1 Feature Discovery with Catboost

As in the predictive analytics framework, gradient boosted machines as implemented with Catboost are used for exploratory data analysis. The framework uses LightGBM, while Catboost is used here. LightGBM does not currently support GPUs under Windows Subsystem for Linux 2, which is why Catboost is used here.

Code
dat.perm.unk %>% 
  filter(IsTraining == TRUE) %>% 
  select(all_of(gbm.pred.cols)) %>% 
  mutate(Sex = as.numeric(Sex)) %>%
  catboost.load_pool(
    data=.,
    label = dat.perm.unk[IsTraining==TRUE,AE_Amount],
    weight = dat.perm.unk[IsTraining==TRUE,ExpDth_Amt_VBT2015]) ->
  train_pool_perm_unk

dat.perm.unk %>% 
  filter(IsTraining == FALSE) %>% 
  select(all_of(gbm.pred.cols)) %>% 
  mutate(Sex = as.numeric(Sex)) %>%
  catboost.load_pool(
    data=.,
    label = dat.perm.unk[IsTraining==FALSE,AE_Amount],
    weight = dat.perm.unk[IsTraining==FALSE,ExpDth_Amt_VBT2015]) ->
  test_pool_perm_unk

if(bUseCache & file.exists(
  'dat.term.mod.perm.unk.cbm'
) & !bInvalidateCaches)
{
  cb.model.perm.unk <- catboost.load_model(model_path = 'dat.term.mod.perm.unk.cbm')
} else {
  cb.model.perm.unk <- catboost.train(learn_pool=train_pool_perm_unk,
                             test_pool = test_pool_perm_unk,
                             params=fit_params)
  
  catboost.save_model(cb.model.perm.unk,
                      model_path = 'dat.term.mod.perm.unk.cbm')
}

8.1.2 Feature Importance

Next is to check for the variables with most variability. Duration, face amount, and attained age top the importance chart.

Code
shp.perm.unk <- shapviz(
  cb.model.perm.unk,
  X=as.data.frame(dat.perm.unk[IsTraining==TRUE,..gbm.pred.cols]),
  X_pred=train_pool_perm_unk
)
setDT(shp.perm.unk$X)

sv_importance(shp.perm.unk) + theme_minimal()

8.1.3 Feature Interactions

Code
imp.int.perm.unk <- catboost.get_feature_importance(
  cb.model.perm.unk,
  type="Interaction"
)



mod.ft.map.perm.unk <- data.table(
  FeatureNames=names(dat.perm.unk[IsTraining==TRUE,..gbm.pred.cols])
)

mod.ft.map.perm.unk[,ID:=1:nrow(.SD)-1]

imp.int.perm.unk %>%
  merge(mod.ft.map.perm.unk,
        by.x="feature1_index",
        by.y="ID") %>%
  merge(mod.ft.map.perm.unk,
        by.x="feature2_index",
        by.y="ID") %>%
  setnames(
    old=c("FeatureNames.x","FeatureNames.y"),
    new=c("Feature1","Feature2")
  ) %>% 
  mutate(feature1_index=NULL,feature2_index=NULL) %>%
  as.data.table() %>%
  setcolorder(c("Feature1","Feature2","score")) -> 
  imp.int.perm.unk

imp.int.perm.unk[order(-score)] %>%
  filter(Feature1 != "Noise") %>%
  flextable() %>%
  colformat_double(j="score",digits=2)

Feature1

Feature2

score

Face_Amount_Band

AA_Grp

23.49

Face_Amount_Band

Dur_Grp

19.03

AA_Grp

Dur_Grp

12.30

Sex

Face_Amount_Band

3.54

Sex

AA_Grp

2.79

Sex

Dur_Grp

1.96

8.1.4 Feature Interaction Plots

Each of the interactions are discussed below with plots using SHAP values from both sides of the interaction.

8.1.4.1 Attained Age Group

8.1.4.1.1 Face Amount Band SHAPs

Several things are apparent in the following ridge plot:

  1. The two lowest face amount bands, <10K and 10k - <25K, tend to be strongly separated rightward from the others.
  2. Starting at about attained age 61 or so, the difference between those two low bands and the others narrows.
  3. This suggests the potential that the slope of the 2015 VBT above attained age 65 for unknown perm business is generally off.
Code
data.table(
  cbind(
    shp.perm.unk$X[,.(Face_Amount_Band,
                      AA_Grp)],
    shap=shp.perm.unk$S[,"Face_Amount_Band"]
  )
) %>%
  ggplot(aes(y=AA_Grp, x=exp(shap),color=Face_Amount_Band,fill=Face_Amount_Band)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,NA)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("by attained age")) +
  theme_minimal()

8.1.4.1.2 Attained Age SHAPs

The interaction within attained age group SHAPs is harder to trace out. Higher attained ages 61+ exhibit different locations of the SHAP distribution for lower face amounts than higher. This is borne out in both views below.

Code
plot.lbls.aa <- dat.perm.unk[,levels(AA_Grp)]
plot.lbls.aa[-(1:5)*4] <- ""

data.table(
  cbind(
    shp.perm.unk$X[,.(Face_Amount_Band,
                      AA_Grp)],
    shap=shp.perm.unk$S[,"AA_Grp"]
  )
) %>%
  ggplot(aes(y=AA_Grp, x=exp(shap),color=Face_Amount_Band,fill=Face_Amount_Band)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,NA)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  scale_y_discrete(labels=plot.lbls.aa) +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect AA_Grp"),
          subtitle = paste0("by attained age")) +
  theme_minimal() + 
  facet_wrap(vars(Face_Amount_Band))

Code
plot.lbls.fa <- dat.perm.unk[,levels(Face_Amount_Band)]
plot.lbls.fa[-c(1,6,11)] <- ""

data.table(
  cbind(
    shp.perm.unk$X[,.(Face_Amount_Band,
                      AA_Grp)],
    shap=shp.perm.unk$S[,"AA_Grp"]
  )
) %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=AA_Grp,fill=AA_Grp)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,NA)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  scale_y_discrete(labels=plot.lbls.fa) +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect AA_Grp"),
          subtitle = paste0("by face amount band")) +
  theme_minimal() +
  facet_wrap(vars(AA_Grp))

8.1.4.2 Duration Group

8.1.4.2.1 Face Amount Band SHAPs

There is some suggestion of differing slope by duration across face amount bands.

Code
data.table(
  cbind(
    shp.perm.unk$X[,.(Face_Amount_Band,
                      Dur_Grp)],
    shap=shp.perm.unk$S[,"Face_Amount_Band"]
  )
) %>%
  ggplot(aes(y=Dur_Grp, x=exp(shap),color=Face_Amount_Band,fill=Face_Amount_Band)) +
  stat_density_ridges(alpha=0.9,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,NA)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("by face amount and duration")) +
  theme_minimal() +
  facet_wrap(vars(Face_Amount_Band))

8.1.4.2.2 Duration Group SHAPs

There are at least two sources of interaction here:

  1. Early duration and low face amount
  2. Multimodal distributions, with a diversity in the number and location of modes
Code
data.table(
  cbind(
    shp.perm.unk$X[,.(Face_Amount_Band,
                      Dur_Grp)],
    shap=shp.perm.unk$S[,"Dur_Grp"]
  )
) %>%
  ggplot(aes(y=Dur_Grp, x=exp(shap),color=Face_Amount_Band,fill=Face_Amount_Band)) +
  stat_density_ridges(alpha=0.9,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,NA)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Dur_Grp"),
          subtitle = paste0("by face amount and duration")) +
  theme_minimal() +
  facet_wrap(vars(Face_Amount_Band))

8.1.4.3 Attained Age vs Duration

8.1.4.3.1 Attained Age SHAPs

There appears to be an interaction for durations 16+, attained ages 61+, where the attained age SHAP is lower than for others.

Code
data.table(
  cbind(
    shp.perm.unk$X[,.(AA_Grp,
                    Dur_Grp)],
    shap=shp.perm.unk$S[,"AA_Grp"]
  )
) %>%
  ggplot(aes(y=AA_Grp, x=exp(shap),color=Dur_Grp,fill=Dur_Grp)) +
  stat_density_ridges(alpha=0.9,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,NA)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  scale_y_discrete(labels=plot.lbls.aa) +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect AA_Grp")) +
  theme_minimal() +
  facet_wrap(vars(Dur_Grp))

8.1.4.3.2 Duration SHAPs

The opposite is true within the duration SHAPs. Here, there is an interaction for early durations <16 for attained ages 61+.

Code
data.table(
  cbind(
    shp.perm.unk$X[,.(AA_Grp,
                      Dur_Grp)],
    shap=shp.perm.unk$S[,"Dur_Grp"]
  )
) %>%
  ggplot(aes(y=AA_Grp, x=exp(shap),color=Dur_Grp,fill=Dur_Grp)) +
  stat_density_ridges(alpha=0.9,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,NA)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  scale_y_discrete(labels=plot.lbls.aa) +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Dur_Grp")) +
  theme_minimal() +
  facet_wrap(vars(Dur_Grp))

Overall, this suggests that the current dataset and 2015 VBT may not have slopes in sync at older ages.

8.1.5 Next Steps in Modeling

The catboost modeling is informative here for exploratory analytics. We next build an elastic net model which includes the main effects plus the top eight interactions from the catboost analysis. This cutoff is chosen arbitrarily. The top three are stronger than the others, while the next five seem to cluster in the middle. These interactions may turn out to be unneeded. Elastic net modeling will reveal which is truly important in a mortality model.

While the work for supporting this is omitted here, it became clear that separate models should be fit for above and below 100K face amounts. A unified model had difficulty adequately fitting face amounts below 100K.

8.1.6 Elastic Net Models

The following formula is used below:

Code
if(bUseAllInteractions) {
  glmnetFormula <- as.formula(
    paste0(
      "~ -1 + (",
      paste(pred.cols,collapse=" + "),
      ")^2"
    )
)
} else {
  glmnetFormula <- as.formula(
    paste(c("~ -1",
            pred.cols,
            imp.int.perm.unk[order(-score)][Feature1 != "Noise",
                           paste0(Feature1,":",Feature2)]),
          collapse=" + ")
  )
}


print(glmnetFormula)
~-1 + Sex + Face_Amount_Band + AA_Grp + Dur_Grp + Face_Amount_Band:AA_Grp + 
    Face_Amount_Band:Dur_Grp + AA_Grp:Dur_Grp + Sex:Face_Amount_Band + 
    Sex:AA_Grp + Sex:Dur_Grp

The interactions included in the model are as follows:

Code
data.table(Interaction=imp.int.perm.unk[order(-score)][Feature1 != "Noise",
                         paste0(Feature1,":",Feature2)]) %>% 
  flextable()

Interaction

Face_Amount_Band:AA_Grp

Face_Amount_Band:Dur_Grp

AA_Grp:Dur_Grp

Sex:Face_Amount_Band

Sex:AA_Grp

Sex:Dur_Grp

8.1.6.1 Model Fit

Code
dat.perm.unk.el <- prepELData(
  formula=glmnetFormula,
  data=dat.perm.unk,
  predictors = pred.cols,
  response = "AE_Amount",
  weights = "ExpDth_Amt_VBT2015",
  useSparse = T
)


set.seed(cvfit.seed)
if(bUseCache & file.exists(
  'perm.model.unk.el.rds'
) & !bInvalidateCaches)
{
  cvfit.perm.unk <- readRDS('perm.model.unk.el.rds')
} else
{
  cvfit.perm.unk <- fitCVGLMNet(
    dat.perm.unk.el,
    nfolds = nFolds
  )
  
  if(bUseCache)
    saveRDS(cvfit.perm.unk, 'perm.model.unk.el.rds')
}
8.1.6.1.1 Usual plots for Elastic Net Models

When presenting elastic net models, the cross validation plot for \(\lambda\) and the coefficient shrinkage plots are provided.

At the minimum \(\lambda\) of 0.0001798, the model has 174 parameters.

Code
plot(cvfit.perm.unk)

Code
plot(cvfit.perm.unk$glmnet.fit,xvar="lambda")

8.1.6.1.2 Factor Table

The exponentiated coefficients are as follows:

Code
reformatCoefs(cvfit.perm.unk, pred.cols)  %>%
  filter(Coef != 0) %>%
  select(Feature1Name,
         Feature1Level,
         Feature2Name,
         Feature2Level,
         Coef) %>%
  mutate(Coef=exp(Coef)) %>%
  flextable() %>%
  set_formatter(
    Coef=function(x) paste0(sprintf("%.01f", 100*x),"%")
  ) %>%
  theme_vanilla()

Feature1Name

Feature1Level

Feature2Name

Feature2Level

Coef

(Intercept)

(Intercept)

133.8%

Sex

F

99.7%

Sex

M

101.2%

Face_Amount_Band

25K - <50K

83.8%

Face_Amount_Band

50K - <100K

79.2%

Face_Amount_Band

100K - <250K

67.6%

Face_Amount_Band

250K - <500K

76.1%

Face_Amount_Band

500K - <1M

64.8%

Face_Amount_Band

1M - <2.5M

57.0%

AA_Grp

18-25

95.9%

AA_Grp

26-30

104.2%

AA_Grp

31-35

107.1%

AA_Grp

41-45

108.2%

AA_Grp

46-50

127.8%

AA_Grp

51-55

133.6%

AA_Grp

56-60

127.1%

AA_Grp

61-65

122.5%

AA_Grp

66-70

112.7%

AA_Grp

71-75

109.8%

AA_Grp

76-80

103.6%

AA_Grp

81-85

99.1%

AA_Grp

86-90

95.5%

AA_Grp

91-95

93.3%

AA_Grp

96-100

96.3%

AA_Grp

101-105

100.2%

Dur_Grp

6-10

115.2%

Dur_Grp

11-15

117.1%

Dur_Grp

16-20

106.5%

Dur_Grp

21-30

97.9%

Dur_Grp

31-120

81.1%

Face_Amount_Band

10K - <25K

AA_Grp

18-25

97.8%

Face_Amount_Band

10M+

AA_Grp

18-25

163.3%

Face_Amount_Band

25K - <50K

AA_Grp

31-35

102.5%

Face_Amount_Band

50K - <100K

AA_Grp

31-35

101.6%

Face_Amount_Band

250K - <500K

AA_Grp

31-35

95.1%

Face_Amount_Band

1M - <2.5M

AA_Grp

36-40

107.8%

Face_Amount_Band

10K - <25K

AA_Grp

46-50

102.3%

Face_Amount_Band

50K - <100K

AA_Grp

46-50

96.0%

Face_Amount_Band

25K - <50K

AA_Grp

51-55

85.4%

Face_Amount_Band

50K - <100K

AA_Grp

51-55

87.6%

Face_Amount_Band

100K - <250K

AA_Grp

51-55

86.3%

Face_Amount_Band

10K - <25K

AA_Grp

56-60

99.7%

Face_Amount_Band

25K - <50K

AA_Grp

56-60

87.1%

Face_Amount_Band

50K - <100K

AA_Grp

56-60

77.4%

Face_Amount_Band

100K - <250K

AA_Grp

56-60

87.0%

Face_Amount_Band

25K - <50K

AA_Grp

61-65

94.5%

Face_Amount_Band

50K - <100K

AA_Grp

61-65

85.6%

Face_Amount_Band

100K - <250K

AA_Grp

61-65

91.4%

Face_Amount_Band

250K - <500K

AA_Grp

61-65

96.9%

Face_Amount_Band

500K - <1M

AA_Grp

61-65

79.6%

Face_Amount_Band

25K - <50K

AA_Grp

66-70

98.3%

Face_Amount_Band

50K - <100K

AA_Grp

66-70

90.5%

Face_Amount_Band

100K - <250K

AA_Grp

66-70

97.6%

Face_Amount_Band

250K - <500K

AA_Grp

66-70

85.2%

Face_Amount_Band

500K - <1M

AA_Grp

66-70

85.9%

Face_Amount_Band

1M - <2.5M

AA_Grp

66-70

58.9%

Face_Amount_Band

2.5M - <5M

AA_Grp

66-70

75.9%

Face_Amount_Band

10K - <25K

AA_Grp

71-75

99.3%

Face_Amount_Band

25K - <50K

AA_Grp

71-75

97.8%

Face_Amount_Band

50K - <100K

AA_Grp

71-75

93.3%

Face_Amount_Band

100K - <250K

AA_Grp

71-75

98.6%

Face_Amount_Band

250K - <500K

AA_Grp

71-75

88.2%

Face_Amount_Band

500K - <1M

AA_Grp

71-75

102.4%

Face_Amount_Band

1M - <2.5M

AA_Grp

71-75

95.1%

Face_Amount_Band

2.5M - <5M

AA_Grp

71-75

59.7%

Face_Amount_Band

10K - <25K

AA_Grp

76-80

97.4%

Face_Amount_Band

25K - <50K

AA_Grp

76-80

99.0%

Face_Amount_Band

100K - <250K

AA_Grp

76-80

107.0%

Face_Amount_Band

500K - <1M

AA_Grp

76-80

92.2%

Face_Amount_Band

1M - <2.5M

AA_Grp

76-80

89.7%

Face_Amount_Band

2.5M - <5M

AA_Grp

76-80

82.2%

Face_Amount_Band

10M+

AA_Grp

76-80

88.0%

Face_Amount_Band

10K - <25K

AA_Grp

81-85

97.9%

Face_Amount_Band

25K - <50K

AA_Grp

81-85

101.3%

Face_Amount_Band

50K - <100K

AA_Grp

81-85

106.0%

Face_Amount_Band

100K - <250K

AA_Grp

81-85

117.4%

Face_Amount_Band

250K - <500K

AA_Grp

81-85

100.0%

Face_Amount_Band

500K - <1M

AA_Grp

81-85

105.1%

Face_Amount_Band

1M - <2.5M

AA_Grp

81-85

93.5%

Face_Amount_Band

10K - <25K

AA_Grp

86-90

100.1%

Face_Amount_Band

25K - <50K

AA_Grp

86-90

106.8%

Face_Amount_Band

50K - <100K

AA_Grp

86-90

112.9%

Face_Amount_Band

100K - <250K

AA_Grp

86-90

126.9%

Face_Amount_Band

250K - <500K

AA_Grp

86-90

113.1%

Face_Amount_Band

1M - <2.5M

AA_Grp

86-90

133.4%

Face_Amount_Band

2.5M - <5M

AA_Grp

86-90

152.0%

Face_Amount_Band

10K - <25K

AA_Grp

91-95

104.3%

Face_Amount_Band

25K - <50K

AA_Grp

91-95

113.8%

Face_Amount_Band

50K - <100K

AA_Grp

91-95

124.4%

Face_Amount_Band

100K - <250K

AA_Grp

91-95

138.9%

Face_Amount_Band

250K - <500K

AA_Grp

91-95

126.5%

Face_Amount_Band

500K - <1M

AA_Grp

91-95

125.8%

Face_Amount_Band

1M - <2.5M

AA_Grp

91-95

111.1%

Face_Amount_Band

10K - <25K

AA_Grp

96-100

104.6%

Face_Amount_Band

25K - <50K

AA_Grp

96-100

115.9%

Face_Amount_Band

50K - <100K

AA_Grp

96-100

129.0%

Face_Amount_Band

100K - <250K

AA_Grp

96-100

138.9%

Face_Amount_Band

250K - <500K

AA_Grp

96-100

128.4%

Face_Amount_Band

500K - <1M

AA_Grp

96-100

109.4%

Face_Amount_Band

1M - <2.5M

AA_Grp

96-100

159.7%

Face_Amount_Band

2.5M - <5M

AA_Grp

96-100

151.2%

Face_Amount_Band

5M - <10M

AA_Grp

96-100

323.2%

Face_Amount_Band

500K - <1M

AA_Grp

101-105

100.6%

Face_Amount_Band

2.5M - <5M

AA_Grp

101-105

105.9%

Face_Amount_Band

2.5M - <5M

Dur_Grp

2-2

112.9%

Face_Amount_Band

10K - <25K

Dur_Grp

6-10

102.5%

Face_Amount_Band

100K - <250K

Dur_Grp

11-15

88.4%

Face_Amount_Band

250K - <500K

Dur_Grp

11-15

87.6%

Face_Amount_Band

10M+

Dur_Grp

11-15

267.1%

Face_Amount_Band

250K - <500K

Dur_Grp

16-20

97.2%

Face_Amount_Band

1M - <2.5M

Dur_Grp

16-20

97.2%

Face_Amount_Band

10M+

Dur_Grp

16-20

263.7%

Face_Amount_Band

10K - <25K

Dur_Grp

21-30

90.7%

Face_Amount_Band

25K - <50K

Dur_Grp

21-30

94.5%

Face_Amount_Band

50K - <100K

Dur_Grp

21-30

93.9%

Face_Amount_Band

250K - <500K

Dur_Grp

21-30

101.5%

Face_Amount_Band

500K - <1M

Dur_Grp

21-30

99.4%

Face_Amount_Band

2.5M - <5M

Dur_Grp

21-30

71.6%

Face_Amount_Band

10K - <25K

Dur_Grp

31-120

93.2%

Face_Amount_Band

50K - <100K

Dur_Grp

31-120

100.3%

Face_Amount_Band

100K - <250K

Dur_Grp

31-120

109.9%

Face_Amount_Band

250K - <500K

Dur_Grp

31-120

104.3%

Face_Amount_Band

500K - <1M

Dur_Grp

31-120

119.6%

Face_Amount_Band

1M - <2.5M

Dur_Grp

31-120

140.2%

Face_Amount_Band

10M+

Dur_Grp

31-120

59.5%

AA_Grp

18-25

Dur_Grp

6-10

96.4%

AA_Grp

71-75

Dur_Grp

6-10

114.9%

AA_Grp

76-80

Dur_Grp

6-10

101.4%

AA_Grp

81-85

Dur_Grp

6-10

107.7%

AA_Grp

18-25

Dur_Grp

11-15

103.4%

AA_Grp

66-70

Dur_Grp

11-15

102.3%

AA_Grp

71-75

Dur_Grp

11-15

110.2%

AA_Grp

76-80

Dur_Grp

11-15

101.3%

AA_Grp

18-25

Dur_Grp

16-20

88.9%

AA_Grp

26-30

Dur_Grp

16-20

99.9%

AA_Grp

66-70

Dur_Grp

16-20

105.1%

AA_Grp

71-75

Dur_Grp

16-20

103.6%

AA_Grp

91-95

Dur_Grp

16-20

99.1%

AA_Grp

18-25

Dur_Grp

21-30

114.7%

AA_Grp

26-30

Dur_Grp

21-30

120.3%

AA_Grp

56-60

Dur_Grp

21-30

100.3%

AA_Grp

66-70

Dur_Grp

21-30

99.8%

AA_Grp

71-75

Dur_Grp

21-30

99.6%

AA_Grp

81-85

Dur_Grp

21-30

94.6%

AA_Grp

86-90

Dur_Grp

21-30

89.7%

AA_Grp

31-35

Dur_Grp

31-120

117.6%

AA_Grp

46-50

Dur_Grp

31-120

105.1%

AA_Grp

56-60

Dur_Grp

31-120

92.6%

AA_Grp

61-65

Dur_Grp

31-120

83.5%

AA_Grp

66-70

Dur_Grp

31-120

84.3%

AA_Grp

71-75

Dur_Grp

31-120

88.6%

AA_Grp

76-80

Dur_Grp

31-120

95.8%

Sex

M

Face_Amount_Band

10K - <25K

108.1%

Sex

M

Face_Amount_Band

25K - <50K

106.1%

Sex

M

Face_Amount_Band

50K - <100K

104.7%

Sex

M

Face_Amount_Band

100K - <250K

102.4%

Sex

M

Face_Amount_Band

250K - <500K

98.7%

Sex

M

Face_Amount_Band

1M - <2.5M

93.0%

Sex

M

Face_Amount_Band

2.5M - <5M

69.4%

Sex

M

Face_Amount_Band

5M - <10M

57.7%

Sex

M

Face_Amount_Band

10M+

150.4%

Sex

M

AA_Grp

31-35

104.0%

Sex

M

AA_Grp

51-55

111.4%

Sex

M

AA_Grp

56-60

111.2%

Sex

M

AA_Grp

61-65

105.7%

Sex

M

AA_Grp

66-70

108.8%

Sex

M

AA_Grp

71-75

102.5%

Sex

M

AA_Grp

81-85

99.7%

Sex

M

AA_Grp

86-90

96.1%

Sex

M

AA_Grp

91-95

96.4%

Sex

M

AA_Grp

96-100

100.3%

Sex

M

AA_Grp

101-105

107.6%

Sex

M

Dur_Grp

16-20

101.1%

Sex

M

Dur_Grp

21-30

102.9%

Note that interaction terms must be considered together, as noted in prior sections. Since penalization is roughly equivalent to Bayesian credibility, these factors are arguably credible. This is true even when dealing with factors with small effect sizes, such as the standalone factor for Duration 3.

Plots of effects are much more digestible.

In the plots and tables that follow, we fix all of the other variables at their middle values when extracting final factors.

We generate some supporting tables: a factor grid for attaching to the experience, and a list of interactions present in the model

Code
dat.perm.unk[,..pred.cols] %>%
  lapply(levels) %>%
  expand.grid() %>%
  setDT() -> 
  dat.perm.unk.grid 

dat.perm.unk.grid %>%
  model.Matrix(
    object=glmnetFormula,
    data=.,
    sparse=T
  ) %>%
  predict(
    cvfit.perm.unk,
    newx=.,
    s="lambda.min"
  ) %>% 
  as.vector() ->
  newCoef

dat.perm.unk.grid %>%
  add_column(
    Factor=exp(newCoef)
  )  %>% 
  setDT() -> 
  dat.perm.unk.grid 

write.xlsx(dat.perm.unk.grid,
           file="dat.perm.unk.grid.xlsx")

reformatCoefs(cvfit.perm.unk, pred.cols) %>%
  filter(Coef != 0 & !is.na(Feature2Name)) %>% 
  select(Feature1Name,Feature2Name) %>% 
  distinct() %>%
  as.list() %>%
  purrr::list_transpose() ->
  perm.unk.int.list
8.1.6.1.3 Plots of Terms

Below are plots of the 2-way interaction terms, with external factors fixed at their middle values.

  1. Attained Age x Face Amount Band: A “ridge” between attained ages 35 and 70 is apparent at the lowest face amounts. The ridge vanishes with increasing face amount, then reappears at higher face amounts.
  2. Duration x Face Amount Band: A similar though smaller “ridge” is observed for durations 6-30.
  3. Attained Age x Duration: Mortality by attained age steepens with increasing age from 81-100, with declining slope by duration at later durations.
  4. Face Amount Band x Sex: This interaction may be misleading. There is very little data at such large face amounts, especially for females. This can be observed in the Goodness-of-Fit section.
  5. Attained Age x Sex: The interaction here appears to be with males aged 46-70, with increased mortality.
8.1.6.1.4 Tables of Terms

Below are tables of the 2-way interaction terms, with external factors fixed at their middle values.

AA_Grp

<10K

10K - <25K

25K - <50K

50K - <100K

100K - <250K

250K - <500K

500K - <1M

1M - <2.5M

2.5M - <5M

5M - <10M

10M+

0-17

133.4%

133.4%

111.8%

105.6%

90.2%

101.5%

86.4%

76.0%

133.4%

133.4%

133.4%

18-25

127.9%

125.1%

107.2%

101.2%

86.5%

97.4%

82.9%

72.9%

127.9%

127.9%

208.9%

26-30

138.9%

138.9%

116.4%

110.0%

93.9%

105.8%

90.0%

79.2%

138.9%

138.9%

138.9%

31-35

142.8%

142.8%

122.7%

114.8%

96.5%

103.4%

92.6%

81.4%

142.8%

142.8%

142.8%

36-40

133.4%

133.4%

111.8%

105.6%

90.2%

101.5%

86.4%

82.0%

133.4%

133.4%

133.4%

41-45

144.4%

144.4%

121.0%

114.3%

97.6%

109.9%

93.6%

82.3%

144.4%

144.4%

144.4%

46-50

170.5%

174.4%

142.9%

129.5%

115.2%

129.8%

110.5%

97.2%

170.5%

170.5%

170.5%

51-55

178.2%

178.2%

127.5%

123.5%

104.0%

135.7%

115.5%

101.6%

178.2%

178.2%

178.2%

56-60

169.6%

169.1%

123.8%

104.0%

99.7%

129.1%

109.9%

96.7%

169.6%

169.6%

169.6%

61-65

163.4%

163.4%

129.4%

110.8%

101.0%

120.5%

84.2%

93.1%

163.4%

163.4%

163.4%

66-70

150.3%

150.3%

123.8%

107.6%

99.1%

97.5%

83.7%

50.5%

114.1%

150.3%

150.3%

71-75

146.4%

145.4%

119.9%

108.1%

97.6%

98.4%

97.2%

79.4%

87.5%

146.4%

146.4%

76-80

138.2%

134.6%

114.6%

109.4%

99.9%

105.2%

82.6%

70.6%

113.6%

138.2%

121.6%

81-85

132.1%

129.3%

112.2%

110.9%

104.9%

100.6%

90.0%

70.4%

132.1%

132.1%

132.1%

86-90

127.4%

127.5%

114.0%

113.9%

109.3%

109.7%

82.6%

96.9%

193.6%

127.4%

127.4%

91-95

124.5%

129.8%

118.7%

122.6%

116.9%

119.9%

101.5%

78.9%

124.5%

124.5%

124.5%

96-100

128.5%

134.4%

124.8%

131.2%

120.7%

125.6%

91.1%

117.0%

194.3%

415.3%

128.5%

101-105

133.6%

133.6%

112.0%

105.8%

90.3%

101.7%

87.1%

76.2%

141.5%

133.6%

133.6%

106-110

133.4%

133.4%

111.8%

105.6%

90.2%

101.5%

86.4%

76.0%

133.4%

133.4%

133.4%

111-115

133.4%

133.4%

111.8%

105.6%

90.2%

101.5%

86.4%

76.0%

133.4%

133.4%

133.4%

116-120

133.4%

133.4%

111.8%

105.6%

90.2%

101.5%

86.4%

76.0%

133.4%

133.4%

133.4%

Dur_Grp

<10K

10K - <25K

25K - <50K

50K - <100K

100K - <250K

250K - <500K

500K - <1M

1M - <2.5M

2.5M - <5M

5M - <10M

10M+

1-1

163.4%

163.4%

129.4%

110.8%

101.0%

120.5%

84.2%

93.1%

163.4%

163.4%

163.4%

2-2

163.4%

163.4%

129.4%

110.8%

101.0%

120.5%

84.2%

93.1%

184.4%

163.4%

163.4%

3-3

163.4%

163.4%

129.4%

110.8%

101.0%

120.5%

84.2%

93.1%

163.4%

163.4%

163.4%

4-5

163.4%

163.4%

129.4%

110.8%

101.0%

120.5%

84.2%

93.1%

163.4%

163.4%

163.4%

6-10

188.3%

193.0%

149.1%

127.6%

116.3%

138.9%

97.1%

107.3%

188.3%

188.3%

188.3%

11-15

191.4%

191.4%

151.6%

129.8%

104.6%

123.6%

98.7%

109.1%

191.4%

191.4%

511.2%

16-20

174.0%

174.0%

137.9%

118.0%

107.5%

124.7%

89.7%

96.4%

174.0%

174.0%

458.9%

21-30

159.9%

145.1%

119.7%

101.8%

98.8%

119.7%

82.0%

91.2%

114.6%

159.9%

159.9%

31-120

110.7%

103.1%

87.7%

75.3%

75.2%

85.1%

68.2%

88.4%

110.7%

110.7%

65.9%

AA_Grp

1-1

2-2

3-3

4-5

6-10

11-15

16-20

21-30

31-120

0-17

90.2%

90.2%

90.2%

90.2%

103.9%

93.4%

96.0%

88.2%

80.4%

18-25

86.5%

86.5%

86.5%

86.5%

96.0%

92.6%

81.9%

97.0%

77.1%

26-30

93.9%

93.9%

93.9%

93.9%

108.2%

97.3%

100.0%

110.6%

83.7%

31-35

96.5%

96.5%

96.5%

96.5%

111.2%

100.0%

102.8%

94.5%

101.2%

36-40

90.2%

90.2%

90.2%

90.2%

103.9%

93.4%

96.0%

88.2%

80.4%

41-45

97.6%

97.6%

97.6%

97.6%

112.4%

101.1%

103.9%

95.5%

87.0%

46-50

115.2%

115.2%

115.2%

115.2%

132.8%

119.4%

122.7%

112.8%

108.0%

51-55

104.0%

104.0%

104.0%

104.0%

119.8%

107.7%

110.7%

101.7%

92.7%

56-60

99.7%

99.7%

99.7%

99.7%

114.8%

103.2%

106.2%

97.8%

82.3%

61-65

101.0%

101.0%

101.0%

101.0%

116.3%

104.6%

107.5%

98.8%

75.2%

66-70

99.1%

99.1%

99.1%

99.1%

114.2%

105.1%

110.9%

96.8%

74.5%

71-75

97.6%

97.6%

97.6%

97.6%

129.2%

111.5%

107.7%

95.1%

77.1%

76-80

99.9%

99.9%

99.9%

99.9%

116.7%

104.9%

106.4%

97.8%

85.3%

81-85

104.9%

104.9%

104.9%

104.9%

130.1%

108.6%

111.7%

97.1%

93.5%

86-90

109.3%

109.3%

109.3%

109.3%

125.9%

113.2%

116.4%

95.9%

97.5%

91-95

116.9%

116.9%

116.9%

116.9%

134.7%

121.1%

123.3%

114.4%

104.2%

96-100

120.7%

120.7%

120.7%

120.7%

139.0%

125.0%

128.5%

118.1%

107.6%

101-105

90.3%

90.3%

90.3%

90.3%

104.1%

93.5%

96.2%

88.4%

80.5%

106-110

90.2%

90.2%

90.2%

90.2%

103.9%

93.4%

96.0%

88.2%

80.4%

111-115

90.2%

90.2%

90.2%

90.2%

103.9%

93.4%

96.0%

88.2%

80.4%

116-120

90.2%

90.2%

90.2%

90.2%

103.9%

93.4%

96.0%

88.2%

80.4%

Face_Amount_Band

F

M

<10K

163.4%

175.3%

10K - <25K

163.4%

189.4%

25K - <50K

129.4%

147.4%

50K - <100K

110.8%

124.4%

100K - <250K

101.0%

110.9%

250K - <500K

120.5%

127.6%

500K - <1M

84.2%

90.4%

1M - <2.5M

93.1%

92.9%

2.5M - <5M

163.4%

121.7%

5M - <10M

163.4%

101.1%

10M+

163.4%

263.6%

AA_Grp

F

M

0-17

90.2%

93.8%

18-25

86.5%

89.9%

26-30

93.9%

97.7%

31-35

96.5%

104.4%

36-40

90.2%

93.8%

41-45

97.6%

101.5%

46-50

115.2%

119.8%

51-55

104.0%

120.4%

56-60

99.7%

115.3%

61-65

101.0%

110.9%

66-70

99.1%

112.2%

71-75

97.6%

104.1%

76-80

99.9%

103.9%

81-85

104.9%

108.7%

86-90

109.3%

109.2%

91-95

116.9%

117.1%

96-100

120.7%

125.9%

101-105

90.3%

101.1%

106-110

90.2%

93.8%

111-115

90.2%

93.8%

116-120

90.2%

93.8%

Dur_Grp

F

M

1-1

101.0%

110.9%

2-2

101.0%

110.9%

3-3

101.0%

110.9%

4-5

101.0%

110.9%

6-10

116.3%

127.8%

11-15

104.6%

114.9%

16-20

107.5%

119.5%

21-30

98.8%

111.7%

31-120

75.2%

82.6%

8.1.6.2 Goodness-of-Fit

Next comes checking goodness-of-fit, on both a univariate and bivariate basis. Tables are provided which show the ratio of the actual to model predicted deaths and show the associated death counts.

Code
dat.perm.unk.grid%>%
  right_join(dat.perm.unk) %>%
  mutate(ExpDth_Amt_GLMNet=ExpDth_Amt_VBT2015*Factor) %>%
  mutate(Factor=NULL) ->
  dat.perm.unk
8.1.6.2.1 Univariate Fit Checks

Sex

Death Count

Actual-to-Model

F

794,865

100.0%

M

1,603,802

100.0%

Face_Amount_Band

Death Count

Actual-to-Model

<10K

1,627,430

100.4%

10K - <25K

539,301

100.0%

25K - <50K

146,377

99.9%

50K - <100K

56,491

99.9%

100K - <250K

23,896

99.9%

250K - <500K

3,755

99.7%

500K - <1M

1,027

99.4%

1M - <2.5M

337

99.2%

2.5M - <5M

43

98.5%

5M - <10M

8

97.9%

10M+

2

108.8%

AA_Grp

Death Count

Actual-to-Model

0-17

1,400

80.9%

18-25

3,782

98.5%

26-30

3,107

102.5%

31-35

4,193

102.7%

36-40

5,280

101.3%

41-45

7,850

102.6%

46-50

14,261

102.2%

51-55

26,508

101.2%

56-60

47,738

100.6%

61-65

80,826

100.3%

66-70

129,766

100.2%

71-75

199,714

100.1%

76-80

312,711

100.1%

81-85

482,022

100.0%

86-90

571,444

100.0%

91-95

396,735

99.9%

96-100

103,091

99.8%

101-105

7,774

103.2%

106-110

410

47.6%

111-115

48

41.9%

116-120

7

32.7%

Dur_Grp

Death Count

Actual-to-Model

1-1

438

98.9%

2-2

365

103.1%

3-3

374

84.0%

4-5

992

96.1%

6-10

6,847

103.5%

11-15

18,202

102.0%

16-20

29,409

101.4%

21-30

127,489

99.8%

31-120

2,214,551

100.0%

8.1.6.2.2 Bivariate Fit Checks

Face_Amount_Band

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

<10K

645,414

101.0%

982,016

100.0%

10K - <25K

118,995

99.9%

420,306

100.0%

25K - <50K

20,402

99.1%

125,975

100.1%

50K - <100K

6,596

98.6%

49,895

100.1%

100K - <250K

2,846

98.3%

21,050

100.1%

250K - <500K

459

99.5%

3,296

99.8%

500K - <1M

107

95.3%

920

99.9%

1M - <2.5M

39

97.6%

298

99.4%

2.5M - <5M

6

106.9%

37

97.4%

5M - <10M

1

212.5%

7

93.1%

10M+

0

0.0%

2

120.8%

AA_Grp

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

0-17

540

83.5%

860

79.4%

18-25

954

96.9%

2,828

99.0%

26-30

888

108.5%

2,219

100.3%

31-35

1,269

100.3%

2,924

103.7%

36-40

1,743

102.6%

3,537

100.7%

41-45

2,470

108.8%

5,380

100.2%

46-50

4,396

103.8%

9,865

101.6%

51-55

7,765

100.7%

18,743

101.4%

56-60

14,099

100.4%

33,639

100.7%

61-65

24,558

100.3%

56,268

100.3%

66-70

39,042

100.2%

90,724

100.2%

71-75

58,725

100.1%

140,989

100.1%

76-80

95,263

100.2%

217,448

100.1%

81-85

156,136

100.0%

325,886

99.9%

86-90

197,182

100.0%

374,262

99.9%

91-95

142,955

100.0%

253,780

99.9%

96-100

42,679

98.1%

60,412

100.3%

101-105

3,967

99.2%

3,807

104.7%

106-110

208

44.9%

202

49.1%

111-115

22

95.7%

26

22.1%

116-120

4

53.0%

3

20.7%

Dur_Grp

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

1-1

171

94.8%

267

101.8%

2-2

157

131.3%

208

85.4%

3-3

128

87.7%

246

81.9%

4-5

423

112.9%

569

87.2%

6-10

3,872

110.5%

2,975

99.1%

11-15

11,771

102.3%

6,431

101.7%

16-20

19,920

100.1%

9,489

102.5%

21-30

79,411

99.1%

48,078

100.3%

31-120

679,012

99.9%

1,535,539

100.0%

AA_Grp

Face_Amount_Band: <10K

Face_Amount_Band: 10K - <25K

Face_Amount_Band: 25K - <50K

Face_Amount_Band: 50K - <100K

Face_Amount_Band: 100K - <250K

Face_Amount_Band: 250K - <500K

Face_Amount_Band: 500K - <1M

Face_Amount_Band: 1M - <2

Face_Amount_Band: 2

Face_Amount_Band: 5M - <10M

Face_Amount_Band: 10M+

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

5M

5M - <5M

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

0-17

45

115.7%

309

60.6%

673

90.6%

213

76.8%

112

70.8%

36

72.5%

9

62.4%

1

17.4%

1

188.8%

0

0.0%

1

317.8%

18-25

291

104.9%

1,246

87.9%

1,425

103.5%

490

105.2%

252

93.7%

56

94.8%

16

110.3%

5

79.9%

0

0.0%

0

0.0%

1

157.4%

26-30

500

114.5%

1,220

99.9%

812

106.6%

364

109.0%

172

111.7%

31

111.6%

6

89.2%

2

79.3%

0

0.0%

0

0.0%

0

0.0%

31-35

1,225

123.3%

1,688

107.1%

793

111.9%

331

114.3%

137

103.6%

14

62.1%

3

55.6%

2

105.4%

0

0.0%

0

0.0%

0

0.0%

36-40

2,378

130.1%

1,912

106.7%

624

98.7%

232

92.6%

115

94.9%

15

80.7%

1

25.8%

3

242.4%

0

0.0%

0

0.0%

41-45

4,381

120.7%

2,464

105.9%

649

96.0%

233

93.0%

108

104.9%

11

87.2%

4

142.5%

0

0.0%

0

0.0%

0

0.0%

46-50

9,340

124.5%

3,746

105.3%

848

95.0%

235

85.1%

79

88.9%

12

129.1%

1

27.9%

0

0.0%

0

0.0%

0

0.0%

51-55

17,200

119.3%

7,169

102.3%

1,553

94.4%

454

90.9%

114

84.0%

14

78.5%

2

49.5%

2

218.2%

0

0.0%

0

0.0%

56-60

29,299

118.3%

13,459

98.7%

3,553

97.6%

1,052

95.5%

323

93.6%

41

93.7%

8

68.3%

3

127.0%

0

0.0%

0

0.0%

61-65

45,383

112.6%

25,156

99.5%

7,063

98.9%

2,354

98.2%

755

97.4%

95

92.2%

13

72.5%

7

102.2%

0

0.0%

0

0.0%

0

0.0%

66-70

71,147

108.7%

41,761

100.3%

11,323

99.3%

3,901

98.9%

1,418

98.7%

176

95.3%

35

88.7%

5

64.5%

0

0.0%

0

0.0%

0

0.0%

71-75

115,269

104.8%

61,200

99.7%

15,373

99.5%

5,334

99.2%

2,117

99.1%

308

97.3%

90

104.5%

22

91.5%

1

44.2%

0

0.0%

76-80

194,394

101.6%

84,970

99.8%

21,616

99.6%

7,794

100.2%

3,259

100.6%

516

101.0%

124

96.8%

33

94.5%

4

82.3%

1

121.3%

0

0.0%

81-85

326,729

99.3%

109,588

99.8%

28,762

100.3%

11,066

100.4%

4,838

100.4%

745

101.0%

223

101.8%

62

97.0%

8

98.2%

1

103.1%

0

0.0%

86-90

416,187

98.4%

106,577

100.2%

29,501

100.3%

12,307

100.3%

5,617

100.4%

906

100.9%

235

100.8%

99

102.4%

13

108.6%

2

106.6%

91-95

305,604

97.8%

61,551

100.3%

17,364

100.5%

7,882

100.6%

3,497

100.7%

588

101.6%

188

102.4%

51

104.0%

8

107.6%

2

85.7%

96-100

81,098

88.9%

14,416

101.3%

4,221

102.1%

2,135

102.3%

944

102.5%

180

105.5%

56

108.2%

34

108.3%

5

129.4%

2

146.0%

101-105

6,556

70.2%

822

89.5%

217

112.8%

109

110.8%

37

126.0%

11

174.1%

13

147.5%

6

125.0%

3

126.7%

106-110

350

31.1%

46

71.1%

7

54.3%

5

203.2%

2

47.9%

0

0.0%

111-115

47

59.1%

1

33.0%

0

0.0%

0

0.0%

116-120

7

32.7%

Face_Amount_Band

Dur_Grp: 1-1

Dur_Grp: 2-2

Dur_Grp: 3-3

Dur_Grp: 4-5

Dur_Grp: 6-10

Dur_Grp: 11-15

Dur_Grp: 16-20

Dur_Grp: 21-30

Dur_Grp: 31-120

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

<10K

91

140.7%

114

163.4%

112

130.3%

470

207.1%

4,800

156.0%

14,815

117.3%

25,467

108.4%

89,995

104.3%

1,491,566

99.8%

10K - <25K

106

140.8%

107

121.0%

114

110.2%

262

110.3%

1,296

119.9%

2,334

101.8%

2,583

106.1%

26,493

99.2%

506,006

100.0%

25K - <50K

163

151.3%

97

121.4%

100

131.8%

165

107.7%

478

100.3%

643

95.1%

851

98.2%

6,952

98.8%

136,928

100.0%

50K - <100K

43

96.6%

24

76.2%

26

86.7%

51

91.2%

150

95.9%

291

95.1%

305

92.7%

2,534

98.3%

53,067

100.1%

100K - <250K

27

108.5%

18

84.8%

15

79.0%

30

84.5%

84

86.2%

93

80.2%

162

94.8%

1,211

100.1%

22,256

100.1%

250K - <500K

6

82.6%

2

41.8%

6

89.9%

10

89.1%

28

87.3%

17

62.8%

30

78.5%

220

103.7%

3,436

100.3%

500K - <1M

1

31.7%

2

96.7%

1

43.6%

3

104.6%

8

79.4%

5

65.8%

9

101.4%

60

94.0%

938

100.5%

1M - <2.5M

1

89.4%

0

0.0%

0

0.0%

1

70.9%

3

82.0%

3

106.2%

1

28.2%

20

93.0%

308

100.8%

2.5M - <5M

0

0.0%

1

1320.6%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

3

74.6%

39

101.7%

5M - <10M

0

0.0%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

1

99.5%

7

103.0%

10M+

0

0.0%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

1

194.0%

1

193.1%

0

0.0%

0

0.0%

AA_Grp

Dur_Grp: 1-1

Dur_Grp: 2-2

Dur_Grp: 3-3

Dur_Grp: 4-5

Dur_Grp: 6-10

Dur_Grp: 11-15

Dur_Grp: 16-20

Dur_Grp: 21-30

Dur_Grp: 31-120

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

0-17

192

90.4%

94

99.8%

82

74.7%

128

72.0%

291

65.8%

348

58.2%

265

124.9%

18-25

39

97.4%

24

45.2%

49

67.8%

105

87.2%

438

90.6%

605

108.7%

1,021

93.0%

1,501

104.9%

26-30

7

246.7%

4

239.4%

3

110.1%

12

231.0%

75

88.7%

420

103.9%

451

84.5%

1,961

104.5%

174

154.0%

31-35

1

204.4%

8

430.7%

5

146.0%

10

183.1%

21

110.5%

121

110.9%

506

95.3%

1,426

99.3%

2,095

109.1%

36-40

3

95.8%

4

46.6%

5

86.2%

11

101.8%

39

213.2%

56

72.6%

167

78.2%

1,573

101.7%

3,422

102.3%

41-45

5

103.7%

8

108.4%

13

276.0%

19

72.1%

67

149.1%

115

121.8%

131

101.3%

1,724

101.8%

5,768

101.9%

46-50

11

123.7%

11

90.9%

17

127.8%

31

72.8%

102

90.1%

181

103.5%

264

134.8%

1,804

100.1%

11,840

102.4%

51-55

13

108.7%

18

142.1%

24

82.3%

52

79.0%

210

101.9%

402

97.8%

424

117.2%

2,372

103.4%

22,993

100.8%

56-60

17

94.3%

28

205.4%

28

92.0%

76

107.1%

358

121.6%

613

116.5%

798

161.2%

3,236

104.8%

42,584

99.4%

61-65

22

166.7%

28

132.2%

26

69.3%

98

114.1%

524

142.9%

945

141.8%

1,130

130.1%

4,761

101.8%

73,292

99.7%

66-70

19

127.0%

25

140.5%

30

121.5%

112

194.8%

782

171.0%

1,408

138.4%

1,726

138.1%

7,498

97.4%

118,166

99.8%

71-75

17

349.1%

18

172.1%

18

59.0%

96

150.9%

927

164.2%

2,381

126.7%

2,510

125.7%

11,536

98.1%

182,211

99.9%

76-80

15

303.1%

17

245.1%

20

262.6%

81

126.2%

1,009

170.7%

3,077

121.9%

4,360

113.7%

18,163

98.8%

285,969

99.9%

81-85

2

429.9%

4

177.3%

5

80.7%

52

143.1%

1,086

165.9%

3,176

99.9%

5,905

93.7%

24,673

98.7%

447,119

100.0%

86-90

3

1747.4%

5

284.2%

10

609.5%

38

369.0%

727

117.7%

2,866

85.6%

5,565

92.2%

26,533

98.5%

535,697

100.0%

91-95

2

459.6%

4

146.1%

2

183.8%

10

34.3%

161

60.1%

1,287

75.7%

3,396

76.5%

15,196

99.5%

376,677

100.1%

96-100

70

107.3%

65

111.8%

37

70.5%

61

295.9%

8

90.8%

199

93.5%

783

67.7%

3,475

91.3%

98,393

100.0%

101-105

22

126.0%

1

30.9%

7

214.7%

57

132.0%

7,687

101.1%

106-110

1

145.9%

0

0.0%

409

50.4%

111-115

0

0.0%

48

41.9%

116-120

7

32.7%

8.2 Perm Model, Smoker Distinct

Code
dat.perm.knw <- readRDS("dat.perm.rds")
dat.perm.knw <- dat.perm.knw[Policies_Exposed > 0]
dat.perm.knw <- dat.perm.knw[!(UW == "U/U/U" &
                               Insurance_Plan == "Perm")]

AA.brks <- c(-1,17,
             seq(25,120,5))
AA.lbls <- paste0(
  AA.brks[1:(length(AA.brks)-1)]+1,
  "-",
  AA.brks[2:length(AA.brks)]
)

Dur.brks <- c(0,1,2,3,5,10,15,20,30,120)
Dur.lbls <- paste0(
  Dur.brks[1:(length(Dur.brks)-1)]+1,
  "-",
  Dur.brks[2:length(Dur.brks)]
)

dat.perm.knw[,`:=`(
  AA_Grp=cut(Attained_Age,
              breaks=AA.brks,
              labels=AA.lbls),
  Dur_Grp=cut(Duration,
               breaks=Dur.brks,
               labels=Dur.lbls)
)]



pred.cols <- names(dat.perm.knw)[c(1,5,6,15:17)]

factor.cols <- pred.cols

dat.perm.knw[,
      (factor.cols):=lapply(.SD,factor),
      .SDcols=factor.cols]

UW.levels<-data.table(UW.levels=dat.perm.knw[,levels(UW)])
UW.levels[,c("NS","Pref_Class","NClasses"):=tstrsplit(UW.levels,"/")]
setkeyv(UW.levels,c("NS","NClasses","Pref_Class"))

set.seed(traintest.seed)
dat.perm.knw %>%
  mutate(
    IsTraining = (runif(nrow(.)) < training.fraction),
    Noise = rnorm(nrow(.)),
    .before = 1
  ) %>% 
  mutate(Face_Amount_Band=fct_relabel(
    Face_Amount_Band,
    #function(.) sub(":"," -",.,fixed=T)
    function(.) fa.remap[Face_Amount_Band.Old==.,Face_Amount_Band.New]
    ),
    AE_Count=Death_Count/ExpDth_Cnt_VBT2015,
    AE_Amount=Death_Claim_Amount/ExpDth_Amt_VBT2015,
    UW=factor(UW,levels=UW.levels$UW.levels)
  ) -> 
  dat.perm.knw

gbm.pred.cols <- c("Noise",pred.cols)

Here is a data preview.

Code
dat.perm.knw %>%
  head(10) %>%
  flextable()

IsTraining

Noise

Sex

Smoker_Status

Attained_Age

Duration

Face_Amount_Band

Insurance_Plan

Number_of_Pfd_Classes

Preferred_Class

Death_Count

Policies_Exposed

ExpDth_Cnt_VBT2015

Death_Claim_Amount

Amount_Exposed

ExpDth_Amt_VBT2015

UW

AA_Grp

Dur_Grp

AE_Count

AE_Amount

TRUE

0.72354004

F

NS

69

6

50K - <100K

Perm

3

3

0

131.846073

0.567626054

0

7,784,406

33,476.09

NS/3/3

66-70

6-10

0.000000

0.000000

TRUE

0.16338468

F

NS

69

6

100K - <250K

Perm

3

1

0

44.808299

0.193062743

0

6,482,107

27,954.57

NS/1/3

66-70

6-10

0.000000

0.000000

TRUE

-0.18609600

F

NS

69

6

100K - <250K

Perm

U

U

1

104.779827

0.463192595

118,240

15,622,740

69,209.88

NS/U/U

66-70

6-10

2.158929

1.708427

TRUE

-0.32956340

F

NS

69

6

250K - <500K

Perm

U

U

1

23.102442

0.102663247

312,493

7,549,700

33,551.89

NS/U/U

66-70

6-10

9.740584

9.313723

TRUE

1.35079832

F

NS

69

6

250K - <500K

Perm

3

1

1

26.586405

0.113763949

250,000

8,384,681

35,867.22

NS/1/3

66-70

6-10

8.790131

6.970154

TRUE

-0.11456002

F

NS

69

6

250K - <500K

Perm

3

3

0

36.353317

0.155361484

0

11,840,441

50,606.81

NS/3/3

66-70

6-10

0.000000

0.000000

FALSE

0.24543130

F

NS

69

6

500K - <1M

Perm

3

1

0

13.228370

0.056639893

0

9,111,743

38,992.50

NS/1/3

66-70

6-10

0.000000

0.000000

TRUE

-0.12631776

F

NS

69

6

500K - <1M

Perm

3

3

0

20.186301

0.086675506

0

12,134,936

52,072.73

NS/3/3

66-70

6-10

0.000000

0.000000

TRUE

-0.07584836

F

NS

69

6

1M - <2.5M

Perm

3

2

0

12.668494

0.054094468

0

16,626,416

70,994.79

NS/2/3

66-70

6-10

0.000000

0.000000

TRUE

0.67148672

F

NS

69

6

2.5M - <5M

Perm

2

1

0

1.906849

0.008142245

0

5,886,563

25,135.62

NS/1/2

66-70

6-10

0.000000

0.000000

8.2.1 Feature Discovery with Catboost

Code
dat.perm.knw %>% 
  filter(IsTraining == TRUE) %>% 
  select(all_of(gbm.pred.cols)) %>% 
  mutate(Sex = as.numeric(Sex)) %>%
  catboost.load_pool(
    data=.,
    label = dat.perm.knw[IsTraining==TRUE,AE_Amount],
    weight = dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]) ->
  train_pool_perm_knw

dat.perm.knw %>% 
  filter(IsTraining == FALSE) %>% 
  select(all_of(gbm.pred.cols)) %>% 
  mutate(Sex = as.numeric(Sex)) %>%
  catboost.load_pool(
    data=.,
    label = dat.perm.knw[IsTraining==FALSE,AE_Amount],
    weight = dat.perm.knw[IsTraining==FALSE,ExpDth_Amt_VBT2015]) ->
  test_pool_perm_knw

if(bUseCache & file.exists(
  'dat.term.mod.perm.knw.cbm'
) & !bInvalidateCaches)
{
  cb.model.perm.knw <- catboost.load_model(model_path = 'dat.term.mod.perm.knw.cbm')
} else {
  cb.model.perm.knw <- catboost.train(learn_pool=train_pool_perm_knw,
                             test_pool = test_pool_perm_knw,
                             params=fit_params)
  
  catboost.save_model(cb.model.perm.knw,
                      model_path = 'dat.term.mod.perm.knw.cbm')
}

8.2.2 Feature Importance

Next is to check for the variables with most variability. It is notable that Sex has mean influence not dissimilar to Noise.

Code
shp.perm.knw <- shapviz(
  cb.model.perm.knw,
  X=as.data.frame(dat.perm.knw[IsTraining==TRUE,..gbm.pred.cols]),
  X_pred=train_pool_perm_knw
)
setDT(shp.perm.knw$X)
shp.perm.knw$X[,c("NS","Pref_Class","NClasses"):=tstrsplit(UW,"/")]

sv_importance(shp.perm.knw) + theme_minimal()

8.2.3 Feature Interactions

The top three interactions are face amount band with each of underwriting, duration, and insurance plan. Sex is notably weakly interacting with other variables.

Code
imp.int.perm.knw <- catboost.get_feature_importance(
  cb.model.perm.knw,
  type="Interaction"
)



mod.ft.map.perm.knw <- data.table(
  FeatureNames=names(dat.perm.knw[IsTraining==TRUE,..gbm.pred.cols])
)

mod.ft.map.perm.knw[,ID:=1:nrow(.SD)-1]

imp.int.perm.knw %>%
  merge(mod.ft.map.perm.knw,
        by.x="feature1_index",
        by.y="ID") %>%
  merge(mod.ft.map.perm.knw,
        by.x="feature2_index",
        by.y="ID") %>%
  setnames(
    old=c("FeatureNames.x","FeatureNames.y"),
    new=c("Feature1","Feature2")
  ) %>% 
  mutate(feature1_index=NULL,feature2_index=NULL) %>%
  as.data.table() %>%
  setcolorder(c("Feature1","Feature2","score")) -> 
  imp.int.perm.knw

imp.int.perm.knw[order(-score)] %>%
  filter(Feature1 != "Noise") %>%
  flextable() %>%
  colformat_double(j="score",digits = 2)

Feature1

Feature2

score

Face_Amount_Band

UW

13.56

Face_Amount_Band

Dur_Grp

8.84

Face_Amount_Band

Insurance_Plan

8.57

Face_Amount_Band

AA_Grp

7.82

Insurance_Plan

UW

5.12

UW

Dur_Grp

4.58

UW

AA_Grp

4.29

Insurance_Plan

Dur_Grp

4.07

Insurance_Plan

AA_Grp

3.26

AA_Grp

Dur_Grp

2.69

Sex

Face_Amount_Band

0.74

Sex

UW

0.57

Sex

Insurance_Plan

0.40

Sex

AA_Grp

0.37

Sex

Dur_Grp

0.34

8.2.4 Feature Interaction Plots

Each of the interactions are discussed below with plots using SHAP values from both sides of the interaction.

8.2.4.1 UW

8.2.4.1.1 Face Amount Band SHAPs

There does not appear to be a meaningful interaction between underwriting and face amount band embedded in the face amount band SHAP values.

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.perm.knw$S[,"Face_Amount_Band"],
    response=dat.perm.knw[IsTraining==TRUE,AE_Amount],
    offset=dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for non-smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.perm.knw$S[,"Face_Amount_Band"],
    response=dat.perm.knw[IsTraining==TRUE,AE_Amount],
    offset=dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '2') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for non-smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.perm.knw$S[,"Face_Amount_Band"],
    response=dat.perm.knw[IsTraining==TRUE,AE_Amount],
    offset=dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '3') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for non-smokers, 3 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.perm.knw$S[,"Face_Amount_Band"],
    response=dat.perm.knw[IsTraining==TRUE,AE_Amount],
    offset=dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '4') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for non-smokers, 4 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.perm.knw$S[,"Face_Amount_Band"],
    response=dat.perm.knw[IsTraining==TRUE,AE_Amount],
    offset=dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.perm.knw$S[,"Face_Amount_Band"],
    response=dat.perm.knw[IsTraining==TRUE,AE_Amount],
    offset=dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == '2') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.perm.knw$S[,"Face_Amount_Band"],
    response=dat.perm.knw[IsTraining==TRUE,AE_Amount],
    offset=dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("for unismokers")) +
  theme_minimal()

8.2.4.1.2 UW SHAPs

The notable interaction here is that the distinction between best and second best of three non-smoker classes appears to collapse below 100K.

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.perm.knw$S[,"UW"],
    response=dat.perm.knw[IsTraining==TRUE,AE_Amount],
    offset=dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.perm.knw$S[,"UW"],
    response=dat.perm.knw[IsTraining==TRUE,AE_Amount],
    offset=dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '2') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.perm.knw$S[,"UW"],
    response=dat.perm.knw[IsTraining==TRUE,AE_Amount],
    offset=dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '3') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, 3 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.perm.knw$S[,"UW"],
    response=dat.perm.knw[IsTraining==TRUE,AE_Amount],
    offset=dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'NS' & NClasses == '4') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for non-smokers, 4 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.perm.knw$S[,"UW"],
    response=dat.perm.knw[IsTraining==TRUE,AE_Amount],
    offset=dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for smokers, unknown preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.perm.knw$S[,"UW"],
    response=dat.perm.knw[IsTraining==TRUE,AE_Amount],
    offset=dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'S' & NClasses == '2') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for smokers, 2 preferred classes")) +
  theme_minimal()

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                    NS,
                    Pref_Class,
                    NClasses)],
    shap=shp.perm.knw$S[,"UW"],
    response=dat.perm.knw[IsTraining==TRUE,AE_Amount],
    offset=dat.perm.knw[IsTraining==TRUE,ExpDth_Amt_VBT2015]
  )
) %>%
  mutate(NS=factor(NS),Pref_Class=factor(Pref_Class))  %>%
  filter(NS == 'U') %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Pref_Class,fill=Pref_Class)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect UW"),
          subtitle = paste0("for unismokers")) +
  theme_minimal()

8.2.4.2 Insurance Plan

8.2.4.2.1 Face Amount Band SHAPs

Outside of the “Other” plan, it is difficult to identify an interaction within the face amount band SHAP values.

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                      Insurance_Plan)],
    shap=shp.perm.knw$S[,"Face_Amount_Band"]
  )
) %>%
  ggplot(aes(y=Insurance_Plan, x=exp(shap),color=Face_Amount_Band,
             fill=Face_Amount_Band)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)") +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Face_Amount_Band"),
          subtitle = paste0("by insurance plan")) +
  theme_minimal()

8.2.4.2.2 Insurance Plan SHAPs

There is some interaction of UL and face amounts under 1M, with UL from 50K - < 1M being markedly higher than the others.

Code
data.table(
  cbind(
    shp.perm.knw$X[,.(Face_Amount_Band,
                      Insurance_Plan)],
    shap=shp.perm.knw$S[,"Insurance_Plan"]
  )
) %>%
  ggplot(aes(y=Face_Amount_Band, x=exp(shap),color=Insurance_Plan,fill=Insurance_Plan)) +
  stat_density_ridges(alpha=0.9/4,scale=1,quantile_lines = T,quantiles = 0.5) + 
  scale_x_continuous(labels = scales::percent,
                     name="Exponentiated SHAP Value (% relative to Offset)",
                     limits = c(NA,1.5)) +
  scale_color_viridis_d() +
  scale_fill_viridis_d() +
  ggtitle(label=paste0("Ridge plot of SHAP values for Effect Insurance_Plan"),
          subtitle = paste0("by face amount band")) +
  theme_minimal()

8.2.5 Next Steps in Modeling

The catboost modeling is informative here for exploratory analytics. We next build an elastic net model which includes the main effects plus the top eight interactions from the catboost analysis. This cutoff is chosen arbitrarily. The top three are stronger than the others, while the next five seem to cluster in the middle. These interactions may turn out to be unneeded. Elastic net modeling will reveal which is truly important in a mortality model.

8.2.6 Elastic Net Models

The following formula is used throughout all non-PLT, term models.

Code
glmnetFormula <- as.formula(
  paste(c("~ -1",
          pred.cols,
          imp.int.perm.knw[order(-score)][Feature1 != "Noise",
                         paste0(Feature1,":",Feature2)]),
        collapse=" + ")
)



print(glmnetFormula)
~-1 + Sex + Face_Amount_Band + Insurance_Plan + UW + AA_Grp + 
    Dur_Grp + Face_Amount_Band:UW + Face_Amount_Band:Dur_Grp + 
    Face_Amount_Band:Insurance_Plan + Face_Amount_Band:AA_Grp + 
    Insurance_Plan:UW + UW:Dur_Grp + UW:AA_Grp + Insurance_Plan:Dur_Grp + 
    Insurance_Plan:AA_Grp + AA_Grp:Dur_Grp + Sex:Face_Amount_Band + 
    Sex:UW + Sex:Insurance_Plan + Sex:AA_Grp + Sex:Dur_Grp

The interactions included in the model are as follows:

Code
data.table(Interaction=imp.int.perm.knw[order(-score)][Feature1 != "Noise",
                         paste0(Feature1,":",Feature2)]) %>% 
  flextable()

Interaction

Face_Amount_Band:UW

Face_Amount_Band:Dur_Grp

Face_Amount_Band:Insurance_Plan

Face_Amount_Band:AA_Grp

Insurance_Plan:UW

UW:Dur_Grp

UW:AA_Grp

Insurance_Plan:Dur_Grp

Insurance_Plan:AA_Grp

AA_Grp:Dur_Grp

Sex:Face_Amount_Band

Sex:UW

Sex:Insurance_Plan

Sex:AA_Grp

Sex:Dur_Grp

8.2.6.1 Model Fit

Code
dat.perm.knw.el <- prepELData(
  formula=glmnetFormula,
  data=dat.perm.knw,
  predictors = pred.cols,
  response = "AE_Amount",
  weights = "ExpDth_Amt_VBT2015",
  #foldid = "FoldID",
  useSparse = T
)


set.seed(cvfit.seed)
if(bUseCache & file.exists(
  'perm.model.knw.el.rds'
) & !bInvalidateCaches)
{
  cvfit.perm.knw <- readRDS('perm.model.knw.el.rds')
} else
{
  cvfit.perm.knw <- fitCVGLMNet(
    dat.perm.knw.el,
    nfolds = 10
  )
  
  if(bUseCache)
    saveRDS(cvfit.perm.knw, 'perm.model.knw.el.rds')
}
8.2.6.1.1 Usual plots for Elastic Net Models

When presenting elastic net models, the cross validation plot for \(\lambda\) and the coefficient shrinkage plots are provided.

At the minimum \(\lambda\) of 0.0002402, the model has 550 parameters.

Code
plot(cvfit.perm.knw)

Code
plot(cvfit.perm.knw$glmnet.fit,xvar="lambda")

8.2.6.1.2 Factor Table

Their exponentiated coefficients are as follows:

Code
reformatCoefs(cvfit.perm.knw, pred.cols)  %>%
  filter(Coef != 0) %>%
  select(Feature1Name,
         Feature1Level,
         Feature2Name,
         Feature2Level,
         Coef) %>%
  mutate(Coef=exp(Coef)) %>%
  flextable() %>%
  set_formatter(
    Coef=function(x) paste0(sprintf("%.01f", 100*x),"%")
  ) %>%
  theme_vanilla()

Feature1Name

Feature1Level

Feature2Name

Feature2Level

Coef

(Intercept)

(Intercept)

94.5%

Sex

F

98.5%

Sex

M

100.1%

Face_Amount_Band

10K - <25K

114.8%

Face_Amount_Band

25K - <50K

113.5%

Face_Amount_Band

50K - <100K

100.1%

Face_Amount_Band

100K - <250K

94.1%

Face_Amount_Band

250K - <500K

89.3%

Face_Amount_Band

500K - <1M

90.0%

Face_Amount_Band

1M - <2.5M

95.8%

Face_Amount_Band

2.5M - <5M

99.3%

Face_Amount_Band

5M - <10M

92.5%

Face_Amount_Band

10M+

71.3%

Insurance_Plan

UL

107.0%

Insurance_Plan

VLSG

96.8%

UW

NS/2/2

132.1%

UW

NS/1/3

93.5%

UW

NS/2/3

97.4%

UW

NS/3/3

128.5%

UW

NS/2/4

77.7%

UW

NS/4/4

117.5%

UW

NS/U/U

117.6%

UW

S/1/2

94.3%

UW

S/2/2

123.7%

UW

S/U/U

118.9%

UW

U/U/U

106.4%

AA_Grp

26-30

120.0%

AA_Grp

31-35

113.7%

AA_Grp

36-40

100.2%

AA_Grp

41-45

106.2%

AA_Grp

46-50

103.9%

AA_Grp

56-60

95.6%

AA_Grp

61-65

97.9%

AA_Grp

71-75

100.6%

AA_Grp

76-80

102.1%

AA_Grp

86-90

103.8%

AA_Grp

96-100

108.0%

AA_Grp

101-105

82.7%

AA_Grp

106-110

62.2%

AA_Grp

111-115

77.2%

AA_Grp

116-120

96.0%

Dur_Grp

4-5

94.8%

Dur_Grp

6-10

96.3%

Dur_Grp

11-15

98.0%

Dur_Grp

16-20

99.0%

Dur_Grp

31-120

92.4%

Face_Amount_Band

50K - <100K

UW

NS/2/2

99.2%

Face_Amount_Band

100K - <250K

UW

NS/2/2

98.9%

Face_Amount_Band

250K - <500K

UW

NS/2/2

100.5%

Face_Amount_Band

500K - <1M

UW

NS/2/2

103.1%

Face_Amount_Band

1M - <2.5M

UW

NS/2/2

107.9%

Face_Amount_Band

2.5M - <5M

UW

NS/2/2

106.2%

Face_Amount_Band

5M - <10M

UW

NS/2/2

94.5%

Face_Amount_Band

10M+

UW

NS/2/2

104.8%

Face_Amount_Band

500K - <1M

UW

NS/1/3

97.3%

Face_Amount_Band

1M - <2.5M

UW

NS/1/3

83.3%

Face_Amount_Band

10M+

UW

NS/1/3

179.4%

Face_Amount_Band

250K - <500K

UW

NS/2/3

100.9%

Face_Amount_Band

2.5M - <5M

UW

NS/2/3

95.5%

Face_Amount_Band

5M - <10M

UW

NS/2/3

92.2%

Face_Amount_Band

10M+

UW

NS/2/3

99.9%

Face_Amount_Band

100K - <250K

UW

NS/3/3

99.4%

Face_Amount_Band

250K - <500K

UW

NS/3/3

99.9%

Face_Amount_Band

500K - <1M

UW

NS/3/3

102.7%

Face_Amount_Band

1M - <2.5M

UW

NS/3/3

105.7%

Face_Amount_Band

2.5M - <5M

UW

NS/3/3

107.0%

Face_Amount_Band

10M+

UW

NS/3/3

113.9%

Face_Amount_Band

10M+

UW

NS/1/4

75.1%

Face_Amount_Band

2.5M - <5M

UW

NS/2/4

75.6%

Face_Amount_Band

5M - <10M

UW

NS/2/4

119.6%

Face_Amount_Band

2.5M - <5M

UW

NS/4/4

132.6%

Face_Amount_Band

10M+

UW

NS/4/4

95.9%

Face_Amount_Band

10K - <25K

UW

NS/U/U

107.8%

Face_Amount_Band

25K - <50K

UW

NS/U/U

102.1%

Face_Amount_Band

50K - <100K

UW

NS/U/U

99.2%

Face_Amount_Band

100K - <250K

UW

NS/U/U

97.0%

Face_Amount_Band

500K - <1M

UW

NS/U/U

99.2%

Face_Amount_Band

2.5M - <5M

UW

NS/U/U

98.8%

Face_Amount_Band

5M - <10M

UW

NS/U/U

93.0%

Face_Amount_Band

10M+

UW

NS/U/U

92.9%

Face_Amount_Band

2.5M - <5M

UW

S/1/2

112.3%

Face_Amount_Band

5M - <10M

UW

S/1/2

87.8%

Face_Amount_Band

500K - <1M

UW

S/2/2

107.3%

Face_Amount_Band

1M - <2.5M

UW

S/2/2

98.7%

Face_Amount_Band

2.5M - <5M

UW

S/2/2

94.7%

Face_Amount_Band

5M - <10M

UW

S/2/2

114.1%

Face_Amount_Band

250K - <500K

UW

S/U/U

100.5%

Face_Amount_Band

500K - <1M

UW

S/U/U

97.7%

Face_Amount_Band

1M - <2.5M

UW

S/U/U

101.0%

Face_Amount_Band

2.5M - <5M

UW

S/U/U

102.7%

Face_Amount_Band

5M - <10M

UW

S/U/U

94.0%

Face_Amount_Band

10M+

UW

S/U/U

114.4%

Face_Amount_Band

50K - <100K

UW

U/U/U

107.9%

Face_Amount_Band

250K - <500K

UW

U/U/U

92.6%

Face_Amount_Band

500K - <1M

UW

U/U/U

98.5%

Face_Amount_Band

1M - <2.5M

UW

U/U/U

73.6%

Face_Amount_Band

10M+

UW

U/U/U

229.0%

Face_Amount_Band

1M - <2.5M

Dur_Grp

2-2

83.1%

Face_Amount_Band

50K - <100K

Dur_Grp

4-5

100.4%

Face_Amount_Band

2.5M - <5M

Dur_Grp

4-5

93.0%

Face_Amount_Band

5M - <10M

Dur_Grp

4-5

88.5%

Face_Amount_Band

10M+

Dur_Grp

4-5

71.7%

Face_Amount_Band

10K - <25K

Dur_Grp

6-10

100.2%

Face_Amount_Band

25K - <50K

Dur_Grp

6-10

103.5%

Face_Amount_Band

100K - <250K

Dur_Grp

6-10

98.3%

Face_Amount_Band

1M - <2.5M

Dur_Grp

6-10

94.5%

Face_Amount_Band

2.5M - <5M

Dur_Grp

6-10

96.0%

Face_Amount_Band

10M+

Dur_Grp

6-10

125.6%

Face_Amount_Band

100K - <250K

Dur_Grp

11-15

98.2%

Face_Amount_Band

1M - <2.5M

Dur_Grp

11-15

99.7%

Face_Amount_Band

2.5M - <5M

Dur_Grp

11-15

95.0%

Face_Amount_Band

5M - <10M

Dur_Grp

11-15

101.4%

Face_Amount_Band

10M+

Dur_Grp

11-15

95.0%

Face_Amount_Band

25K - <50K

Dur_Grp

16-20

98.9%

Face_Amount_Band

50K - <100K

Dur_Grp

16-20

98.2%

Face_Amount_Band

100K - <250K

Dur_Grp

16-20

94.7%

Face_Amount_Band

250K - <500K

Dur_Grp

16-20

99.0%

Face_Amount_Band

1M - <2.5M

Dur_Grp

16-20

105.4%

Face_Amount_Band

2.5M - <5M

Dur_Grp

16-20

102.2%

Face_Amount_Band

10K - <25K

Dur_Grp

21-30

92.5%

Face_Amount_Band

25K - <50K

Dur_Grp

21-30

90.4%

Face_Amount_Band

50K - <100K

Dur_Grp

21-30

92.2%

Face_Amount_Band

100K - <250K

Dur_Grp

21-30

92.9%

Face_Amount_Band

250K - <500K

Dur_Grp

21-30

96.3%

Face_Amount_Band

1M - <2.5M

Dur_Grp

21-30

102.7%

Face_Amount_Band

2.5M - <5M

Dur_Grp

21-30

112.1%

Face_Amount_Band

10M+

Dur_Grp

21-30

101.3%

Face_Amount_Band

10K - <25K

Dur_Grp

31-120

96.5%

Face_Amount_Band

25K - <50K

Dur_Grp

31-120

95.4%

Face_Amount_Band

100K - <250K

Dur_Grp

31-120

100.3%

Face_Amount_Band

250K - <500K

Dur_Grp

31-120

102.9%

Face_Amount_Band

1M - <2.5M

Dur_Grp

31-120

98.5%

Face_Amount_Band

10K - <25K

Insurance_Plan

Perm

101.0%

Face_Amount_Band

100K - <250K

Insurance_Plan

Perm

98.9%

Face_Amount_Band

250K - <500K

Insurance_Plan

Perm

92.9%

Face_Amount_Band

500K - <1M

Insurance_Plan

Perm

88.8%

Face_Amount_Band

1M - <2.5M

Insurance_Plan

Perm

90.7%

Face_Amount_Band

2.5M - <5M

Insurance_Plan

Perm

85.8%

Face_Amount_Band

5M - <10M

Insurance_Plan

Perm

87.9%

Face_Amount_Band

10M+

Insurance_Plan

Perm

187.4%

Face_Amount_Band

25K - <50K

Insurance_Plan

UL

100.2%

Face_Amount_Band

50K - <100K

Insurance_Plan

UL

102.8%

Face_Amount_Band

100K - <250K

Insurance_Plan

UL

101.8%

Face_Amount_Band

250K - <500K

Insurance_Plan

UL

102.0%

Face_Amount_Band

500K - <1M

Insurance_Plan

UL

100.9%

Face_Amount_Band

1M - <2.5M

Insurance_Plan

UL

99.6%

Face_Amount_Band

2.5M - <5M

Insurance_Plan

UL

96.3%

Face_Amount_Band

5M - <10M

Insurance_Plan

UL

93.4%

Face_Amount_Band

50K - <100K

Insurance_Plan

ULSG

102.2%

Face_Amount_Band

500K - <1M

Insurance_Plan

ULSG

98.7%

Face_Amount_Band

1M - <2.5M

Insurance_Plan

ULSG

99.9%

Face_Amount_Band

2.5M - <5M

Insurance_Plan

ULSG

98.9%

Face_Amount_Band

10M+

Insurance_Plan

ULSG

97.7%

Face_Amount_Band

50K - <100K

Insurance_Plan

VL

98.9%

Face_Amount_Band

100K - <250K

Insurance_Plan

VL

97.5%

Face_Amount_Band

250K - <500K

Insurance_Plan

VL

98.0%

Face_Amount_Band

2.5M - <5M

Insurance_Plan

VL

106.5%

Face_Amount_Band

10M+

Insurance_Plan

VL

99.7%

Face_Amount_Band

100K - <250K

Insurance_Plan

VLSG

97.9%

Face_Amount_Band

500K - <1M

Insurance_Plan

VLSG

97.2%

Face_Amount_Band

2.5M - <5M

Insurance_Plan

VLSG

109.7%

Face_Amount_Band

10M+

Insurance_Plan

VLSG

122.4%

Face_Amount_Band

10M+

AA_Grp

18-25

220.9%

Face_Amount_Band

250K - <500K

AA_Grp

36-40

98.5%

Face_Amount_Band

10M+

AA_Grp

36-40

171.5%

Face_Amount_Band

2.5M - <5M

AA_Grp

41-45

109.1%

Face_Amount_Band

10M+

AA_Grp

41-45

99.7%

Face_Amount_Band

25K - <50K

AA_Grp

46-50

108.9%

Face_Amount_Band

100K - <250K

AA_Grp

46-50

97.2%

Face_Amount_Band

250K - <500K

AA_Grp

46-50

99.5%

Face_Amount_Band

500K - <1M

AA_Grp

46-50

98.3%

Face_Amount_Band

10M+

AA_Grp

46-50

135.8%

Face_Amount_Band

10K - <25K

AA_Grp

51-55

116.5%

Face_Amount_Band

25K - <50K

AA_Grp

51-55

114.6%

Face_Amount_Band

50K - <100K

AA_Grp

51-55

106.7%

Face_Amount_Band

500K - <1M

AA_Grp

51-55

99.1%

Face_Amount_Band

2.5M - <5M

AA_Grp

51-55

92.1%

Face_Amount_Band

5M - <10M

AA_Grp

51-55

87.9%

Face_Amount_Band

10M+

AA_Grp

51-55

75.5%

Face_Amount_Band

10K - <25K

AA_Grp

56-60

105.0%

Face_Amount_Band

25K - <50K

AA_Grp

56-60

105.5%

Face_Amount_Band

50K - <100K

AA_Grp

56-60

100.4%

Face_Amount_Band

100K - <250K

AA_Grp

56-60

97.4%

Face_Amount_Band

250K - <500K

AA_Grp

56-60

96.4%

Face_Amount_Band

500K - <1M

AA_Grp

56-60

93.1%

Face_Amount_Band

1M - <2.5M

AA_Grp

56-60

89.8%

Face_Amount_Band

2.5M - <5M

AA_Grp

56-60

86.7%

Face_Amount_Band

10K - <25K

AA_Grp

61-65

107.6%

Face_Amount_Band

25K - <50K

AA_Grp

61-65

105.1%

Face_Amount_Band

50K - <100K

AA_Grp

61-65

103.1%

Face_Amount_Band

100K - <250K

AA_Grp

61-65

98.5%

Face_Amount_Band

250K - <500K

AA_Grp

61-65

93.7%

Face_Amount_Band

500K - <1M

AA_Grp

61-65

93.9%

Face_Amount_Band

1M - <2.5M

AA_Grp

61-65

94.2%

Face_Amount_Band

2.5M - <5M

AA_Grp

61-65

89.3%

Face_Amount_Band

10K - <25K

AA_Grp

66-70

102.1%

Face_Amount_Band

25K - <50K

AA_Grp

66-70

100.5%

Face_Amount_Band

250K - <500K

AA_Grp

66-70

97.4%

Face_Amount_Band

500K - <1M

AA_Grp

66-70

97.2%

Face_Amount_Band

1M - <2.5M

AA_Grp

66-70

98.9%

Face_Amount_Band

5M - <10M

AA_Grp

66-70

102.6%

Face_Amount_Band

100K - <250K

AA_Grp

71-75

101.0%

Face_Amount_Band

500K - <1M

AA_Grp

71-75

101.9%

Face_Amount_Band

1M - <2.5M

AA_Grp

71-75

98.2%

Face_Amount_Band

2.5M - <5M

AA_Grp

71-75

95.8%

Face_Amount_Band

5M - <10M

AA_Grp

71-75

87.1%

Face_Amount_Band

10M+

AA_Grp

71-75

74.2%

Face_Amount_Band

10K - <25K

AA_Grp

76-80

91.6%

Face_Amount_Band

25K - <50K

AA_Grp

76-80

90.3%

Face_Amount_Band

50K - <100K

AA_Grp

76-80

94.4%

Face_Amount_Band

250K - <500K

AA_Grp

76-80

103.7%

Face_Amount_Band

500K - <1M

AA_Grp

76-80

102.8%

Face_Amount_Band

1M - <2.5M

AA_Grp

76-80

103.3%

Face_Amount_Band

10K - <25K

AA_Grp

81-85

88.9%

Face_Amount_Band

25K - <50K

AA_Grp

81-85

91.8%

Face_Amount_Band

50K - <100K

AA_Grp

81-85

99.2%

Face_Amount_Band

100K - <250K

AA_Grp

81-85

106.9%

Face_Amount_Band

250K - <500K

AA_Grp

81-85

113.2%

Face_Amount_Band

500K - <1M

AA_Grp

81-85

112.0%

Face_Amount_Band

1M - <2.5M

AA_Grp

81-85

107.0%

Face_Amount_Band

5M - <10M

AA_Grp

81-85

96.5%

Face_Amount_Band

10M+

AA_Grp

81-85

86.6%

Face_Amount_Band

10K - <25K

AA_Grp

86-90

76.9%

Face_Amount_Band

25K - <50K

AA_Grp

86-90

81.7%

Face_Amount_Band

50K - <100K

AA_Grp

86-90

88.2%

Face_Amount_Band

250K - <500K

AA_Grp

86-90

103.7%

Face_Amount_Band

500K - <1M

AA_Grp

86-90

103.7%

Face_Amount_Band

5M - <10M

AA_Grp

86-90

101.8%

Face_Amount_Band

10M+

AA_Grp

86-90

119.0%

Face_Amount_Band

10K - <25K

AA_Grp

91-95

86.0%

Face_Amount_Band

25K - <50K

AA_Grp

91-95

91.1%

Face_Amount_Band

50K - <100K

AA_Grp

91-95

99.6%

Face_Amount_Band

100K - <250K

AA_Grp

91-95

108.2%

Face_Amount_Band

250K - <500K

AA_Grp

91-95

113.1%

Face_Amount_Band

500K - <1M

AA_Grp

91-95

112.6%

Face_Amount_Band

1M - <2.5M

AA_Grp

91-95

98.6%

Face_Amount_Band

2.5M - <5M

AA_Grp

91-95

103.9%

Face_Amount_Band

10M+

AA_Grp

91-95

99.5%

Face_Amount_Band

100K - <250K

AA_Grp

96-100

102.3%

Face_Amount_Band

500K - <1M

AA_Grp

96-100

106.3%

Face_Amount_Band

10M+

AA_Grp

96-100

76.5%

Insurance_Plan

Perm

UW

NS/2/2

103.0%

Insurance_Plan

UL

UW

NS/2/2

100.7%

Insurance_Plan

ULSG

UW

NS/2/2

100.3%

Insurance_Plan

VL

UW

NS/2/2

99.5%

Insurance_Plan

UL

UW

NS/1/3

87.5%

Insurance_Plan

ULSG

UW

NS/1/3

84.5%

Insurance_Plan

Perm

UW

NS/2/3

108.7%

Insurance_Plan

UL

UW

NS/2/3

96.5%

Insurance_Plan

VLSG

UW

NS/2/3

96.6%

Insurance_Plan

Perm

UW

NS/3/3

116.1%

Insurance_Plan

VL

UW

NS/3/3

110.0%

Insurance_Plan

VLSG

UW

NS/3/3

94.7%

Insurance_Plan

ULSG

UW

NS/1/4

85.8%

Insurance_Plan

UL

UW

NS/2/4

104.0%

Insurance_Plan

ULSG

UW

NS/2/4

89.5%

Insurance_Plan

ULSG

UW

NS/3/4

102.1%

Insurance_Plan

ULSG

UW

NS/4/4

103.0%

Insurance_Plan

Perm

UW

NS/U/U

99.6%

Insurance_Plan

UL

UW

NS/U/U

98.4%

Insurance_Plan

ULSG

UW

NS/U/U

105.7%

Insurance_Plan

VL

UW

NS/U/U

103.5%

Insurance_Plan

VLSG

UW

NS/U/U

105.3%

Insurance_Plan

Perm

UW

S/2/2

106.9%

Insurance_Plan

Perm

UW

S/U/U

102.8%

Insurance_Plan

ULSG

UW

S/U/U

119.3%

Insurance_Plan

VL

UW

S/U/U

96.6%

Insurance_Plan

VLSG

UW

S/U/U

95.8%

Insurance_Plan

ULSG

UW

U/U/U

80.0%

Insurance_Plan

VL

UW

U/U/U

101.0%

Insurance_Plan

VLSG

UW

U/U/U

110.6%

UW

NS/1/4

Dur_Grp

2-2

92.5%

UW

NS/2/2

Dur_Grp

3-3

100.4%

UW

NS/3/3

Dur_Grp

3-3

98.1%

UW

NS/2/4

Dur_Grp

3-3

93.2%

UW

NS/U/U

Dur_Grp

3-3

104.2%

UW

NS/2/2

Dur_Grp

4-5

94.8%

UW

NS/1/3

Dur_Grp

4-5

111.8%

UW

NS/2/3

Dur_Grp

4-5

92.4%

UW

NS/2/4

Dur_Grp

4-5

128.7%

UW

NS/3/4

Dur_Grp

4-5

127.1%

UW

NS/4/4

Dur_Grp

4-5

104.8%

UW

NS/U/U

Dur_Grp

4-5

104.4%

UW

S/2/2

Dur_Grp

4-5

93.3%

UW

NS/2/2

Dur_Grp

6-10

99.3%

UW

NS/1/3

Dur_Grp

6-10

106.3%

UW

NS/3/3

Dur_Grp

6-10

104.5%

UW

NS/U/U

Dur_Grp

6-10

99.0%

UW

S/1/2

Dur_Grp

6-10

89.2%

UW

S/2/2

Dur_Grp

6-10

83.8%

UW

U/U/U

Dur_Grp

6-10

121.0%

UW

NS/2/2

Dur_Grp

11-15

103.2%

UW

NS/1/3

Dur_Grp

11-15

118.2%

UW

NS/2/3

Dur_Grp

11-15

96.6%

UW

NS/3/3

Dur_Grp

11-15

103.6%

UW

NS/3/4

Dur_Grp

11-15

129.7%

UW

NS/U/U

Dur_Grp

11-15

97.6%

UW

S/1/2

Dur_Grp

11-15

91.7%

UW

S/2/2

Dur_Grp

11-15

109.0%

UW

S/U/U

Dur_Grp

11-15

98.3%

UW

NS/2/2

Dur_Grp

16-20

100.6%

UW

NS/1/3

Dur_Grp

16-20

94.9%

UW

NS/2/3

Dur_Grp

16-20

100.3%

UW

NS/3/3

Dur_Grp

16-20

104.1%

UW

NS/U/U

Dur_Grp

16-20

94.0%

UW

S/1/2

Dur_Grp

16-20

108.9%

UW

S/2/2

Dur_Grp

16-20

104.9%

UW

S/U/U

Dur_Grp

16-20

99.1%

UW

NS/2/2

Dur_Grp

21-30

100.7%

UW

NS/2/3

Dur_Grp

21-30

91.1%

UW

NS/3/3

Dur_Grp

21-30

44.6%

UW

NS/4/4

Dur_Grp

21-30

69.8%

UW

S/1/2

Dur_Grp

21-30

106.9%

UW

S/2/2

Dur_Grp

21-30

106.9%

UW

S/U/U

Dur_Grp

21-30

104.1%

UW

U/U/U

Dur_Grp

21-30

100.2%

UW

NS/3/3

Dur_Grp

31-120

58.8%

UW

U/U/U

Dur_Grp

31-120

82.3%

UW

NS/U/U

AA_Grp

18-25

95.5%

UW

NS/1/3

AA_Grp

31-35

93.6%

UW

NS/3/3

AA_Grp

36-40

108.9%

UW

NS/1/3

AA_Grp

41-45

102.0%

UW

NS/2/3

AA_Grp

41-45

101.9%

UW

S/1/2

AA_Grp

41-45

95.2%

UW

NS/2/2

AA_Grp

46-50

106.4%

UW

NS/1/3

AA_Grp

46-50

94.1%

UW

NS/3/3

AA_Grp

46-50

136.4%

UW

NS/U/U

AA_Grp

46-50

100.6%

UW

S/U/U

AA_Grp

46-50

99.0%

UW

NS/3/3

AA_Grp

51-55

111.9%

UW

NS/U/U

AA_Grp

51-55

98.2%

UW

S/U/U

AA_Grp

51-55

95.6%

UW

NS/1/3

AA_Grp

56-60

112.2%

UW

NS/U/U

AA_Grp

56-60

99.1%

UW

S/2/2

AA_Grp

56-60

108.5%

UW

U/U/U

AA_Grp

56-60

110.6%

UW

NS/2/2

AA_Grp

61-65

106.2%

UW

NS/1/3

AA_Grp

61-65

115.2%

UW

NS/3/3

AA_Grp

61-65

117.8%

UW

NS/1/4

AA_Grp

61-65

94.8%

UW

NS/2/4

AA_Grp

61-65

94.9%

UW

NS/4/4

AA_Grp

61-65

107.9%

UW

S/U/U

AA_Grp

61-65

94.9%

UW

U/U/U

AA_Grp

61-65

110.5%

UW

NS/2/2

AA_Grp

66-70

110.1%

UW

NS/1/3

AA_Grp

66-70

78.5%

UW

NS/2/3

AA_Grp

66-70

97.5%

UW

NS/3/3

AA_Grp

66-70

106.6%

UW

NS/1/4

AA_Grp

66-70

95.9%

UW

NS/2/4

AA_Grp

66-70

99.9%

UW

NS/U/U

AA_Grp

66-70

113.4%

UW

S/U/U

AA_Grp

66-70

101.9%

UW

U/U/U

AA_Grp

66-70

133.0%

UW

NS/1/3

AA_Grp

71-75

113.1%

UW

NS/3/3

AA_Grp

71-75

104.6%

UW

NS/4/4

AA_Grp

71-75

114.9%

UW

NS/U/U

AA_Grp

71-75

100.6%

UW

S/1/2

AA_Grp

71-75

104.4%

UW

S/2/2

AA_Grp

71-75

99.9%

UW

U/U/U

AA_Grp

71-75

108.0%

UW

NS/1/3

AA_Grp

76-80

94.3%

UW

NS/2/3

AA_Grp

76-80

86.1%

UW

NS/3/3

AA_Grp

76-80

94.5%

UW

NS/2/4

AA_Grp

76-80

94.9%

UW

NS/3/4

AA_Grp

76-80

113.0%

UW

S/1/2

AA_Grp

76-80

120.7%

UW

S/U/U

AA_Grp

76-80

107.4%

UW

U/U/U

AA_Grp

76-80

103.8%

UW

NS/2/2

AA_Grp

81-85

96.4%

UW

NS/1/3

AA_Grp

81-85

101.1%

UW

NS/2/3

AA_Grp

81-85

97.6%

UW

NS/3/3

AA_Grp

81-85

94.1%

UW

NS/3/4

AA_Grp

81-85

101.3%

UW

S/2/2

AA_Grp

81-85

102.9%

UW

S/U/U

AA_Grp

81-85

111.0%

UW

NS/2/2

AA_Grp

86-90

100.0%

UW

NS/1/3

AA_Grp

86-90

84.6%

UW

NS/2/3

AA_Grp

86-90

89.3%

UW

NS/3/3

AA_Grp

86-90

82.1%

UW

NS/2/4

AA_Grp

86-90

124.8%

UW

NS/3/4

AA_Grp

86-90

74.8%

UW

NS/4/4

AA_Grp

86-90

89.6%

UW

NS/U/U

AA_Grp

86-90

104.6%

UW

S/U/U

AA_Grp

86-90

103.3%

UW

NS/2/2

AA_Grp

91-95

89.4%

UW

NS/1/3

AA_Grp

91-95

90.6%

UW

NS/2/3

AA_Grp

91-95

106.0%

UW

NS/3/3

AA_Grp

91-95

77.9%

UW

NS/4/4

AA_Grp

91-95

87.1%

UW

NS/U/U

AA_Grp

91-95

102.2%

UW

S/U/U

AA_Grp

91-95

85.7%

UW

NS/2/2

AA_Grp

96-100

98.3%

UW

NS/2/3

AA_Grp

96-100

103.4%

UW

NS/3/3

AA_Grp

96-100

81.5%

UW

U/U/U

AA_Grp

96-100

98.7%

UW

U/U/U

AA_Grp

101-105

94.1%

Insurance_Plan

Perm

Dur_Grp

2-2

119.8%

Insurance_Plan

ULSG

Dur_Grp

2-2

90.1%

Insurance_Plan

ULSG

Dur_Grp

4-5

98.1%

Insurance_Plan

VLSG

Dur_Grp

4-5

97.4%

Insurance_Plan

UL

Dur_Grp

6-10

93.3%

Insurance_Plan

ULSG

Dur_Grp

6-10

101.0%

Insurance_Plan

VLSG

Dur_Grp

6-10

98.3%

Insurance_Plan

UL

Dur_Grp

11-15

90.6%

Insurance_Plan

ULSG

Dur_Grp

11-15

98.8%

Insurance_Plan

VLSG

Dur_Grp

11-15

110.5%

Insurance_Plan

Perm

Dur_Grp

16-20

98.3%

Insurance_Plan

UL

Dur_Grp

16-20

96.3%

Insurance_Plan

ULSG

Dur_Grp

16-20

104.0%

Insurance_Plan

Perm

Dur_Grp

21-30

94.7%

Insurance_Plan

UL

Dur_Grp

21-30

103.0%

Insurance_Plan

ULSG

Dur_Grp

31-120

62.7%

Insurance_Plan

VL

AA_Grp

18-25

100.7%

Insurance_Plan

ULSG

AA_Grp

41-45

104.5%

Insurance_Plan

UL

AA_Grp

46-50

107.5%

Insurance_Plan

VL

AA_Grp

46-50

91.6%

Insurance_Plan

Perm

AA_Grp

51-55

99.4%

Insurance_Plan

UL

AA_Grp

51-55

103.4%

Insurance_Plan

ULSG

AA_Grp

51-55

113.2%

Insurance_Plan

VLSG

AA_Grp

51-55

98.9%

Insurance_Plan

UL

AA_Grp

56-60

103.4%

Insurance_Plan

VL

AA_Grp

56-60

98.8%

Insurance_Plan

Perm

AA_Grp

61-65

93.3%

Insurance_Plan

ULSG

AA_Grp

61-65

91.9%

Insurance_Plan

VL

AA_Grp

61-65

103.5%

Insurance_Plan

Perm

AA_Grp

66-70

87.2%

Insurance_Plan

UL

AA_Grp

66-70

92.9%

Insurance_Plan

ULSG

AA_Grp

66-70

98.0%

Insurance_Plan

Perm

AA_Grp

71-75

93.3%

Insurance_Plan

UL

AA_Grp

71-75

103.2%

Insurance_Plan

ULSG

AA_Grp

71-75

92.3%

Insurance_Plan

VL

AA_Grp

71-75

104.3%

Insurance_Plan

VLSG

AA_Grp

71-75

103.2%

Insurance_Plan

UL

AA_Grp

76-80

104.2%

Insurance_Plan

VL

AA_Grp

76-80

96.6%

Insurance_Plan

VLSG

AA_Grp

76-80

99.9%

Insurance_Plan

Perm

AA_Grp

81-85

100.6%

Insurance_Plan

UL

AA_Grp

81-85

102.0%

Insurance_Plan

ULSG

AA_Grp

81-85

99.4%

Insurance_Plan

Perm

AA_Grp

86-90

102.6%

Insurance_Plan

UL

AA_Grp

86-90

97.0%

Insurance_Plan

ULSG

AA_Grp

86-90

100.9%

Insurance_Plan

VL

AA_Grp

86-90

102.1%

Insurance_Plan

Perm

AA_Grp

91-95

101.8%

Insurance_Plan

UL

AA_Grp

91-95

92.4%

Insurance_Plan

ULSG

AA_Grp

91-95

101.6%

Insurance_Plan

VL

AA_Grp

91-95

97.0%

Insurance_Plan

Perm

AA_Grp

96-100

104.0%

Insurance_Plan

ULSG

AA_Grp

96-100

85.1%

Insurance_Plan

VL

AA_Grp

96-100

108.8%

Insurance_Plan

UL

AA_Grp

101-105

82.4%

AA_Grp

41-45

Dur_Grp

2-2

100.4%

AA_Grp

46-50

Dur_Grp

2-2

91.8%

AA_Grp

51-55

Dur_Grp

2-2

78.0%

AA_Grp

61-65

Dur_Grp

2-2

107.1%

AA_Grp

66-70

Dur_Grp

2-2

81.9%

AA_Grp

41-45

Dur_Grp

3-3

107.8%

AA_Grp

56-60

Dur_Grp

3-3

96.5%

AA_Grp

61-65

Dur_Grp

3-3

97.8%

AA_Grp

71-75

Dur_Grp

3-3

98.8%

AA_Grp

46-50

Dur_Grp

4-5

98.6%

AA_Grp

56-60

Dur_Grp

4-5

89.4%

AA_Grp

61-65

Dur_Grp

4-5

97.2%

AA_Grp

71-75

Dur_Grp

4-5

105.9%

AA_Grp

81-85

Dur_Grp

4-5

99.1%

AA_Grp

86-90

Dur_Grp

4-5

105.8%

AA_Grp

91-95

Dur_Grp

4-5

87.9%

AA_Grp

18-25

Dur_Grp

6-10

112.9%

AA_Grp

46-50

Dur_Grp

6-10

111.6%

AA_Grp

51-55

Dur_Grp

6-10

99.6%

AA_Grp

56-60

Dur_Grp

6-10

112.0%

AA_Grp

66-70

Dur_Grp

6-10

96.5%

AA_Grp

71-75

Dur_Grp

6-10

102.4%

AA_Grp

76-80

Dur_Grp

6-10

91.8%

AA_Grp

81-85

Dur_Grp

6-10

94.1%

AA_Grp

86-90

Dur_Grp

6-10

99.1%

AA_Grp

91-95

Dur_Grp

6-10

77.7%

AA_Grp

96-100

Dur_Grp

6-10

119.9%

AA_Grp

46-50

Dur_Grp

11-15

104.0%

AA_Grp

61-65

Dur_Grp

11-15

99.7%

AA_Grp

66-70

Dur_Grp

11-15

98.2%

AA_Grp

71-75

Dur_Grp

11-15

99.0%

AA_Grp

76-80

Dur_Grp

11-15

107.7%

AA_Grp

86-90

Dur_Grp

11-15

91.4%

AA_Grp

91-95

Dur_Grp

11-15

98.8%

AA_Grp

96-100

Dur_Grp

11-15

104.1%

AA_Grp

56-60

Dur_Grp

16-20

99.6%

AA_Grp

66-70

Dur_Grp

16-20

103.1%

AA_Grp

76-80

Dur_Grp

16-20

98.8%

AA_Grp

81-85

Dur_Grp

16-20

99.9%

AA_Grp

86-90

Dur_Grp

16-20

102.8%

AA_Grp

91-95

Dur_Grp

16-20

108.4%

AA_Grp

101-105

Dur_Grp

16-20

110.4%

AA_Grp

41-45

Dur_Grp

21-30

96.6%

AA_Grp

46-50

Dur_Grp

21-30

99.8%

AA_Grp

56-60

Dur_Grp

21-30

99.7%

AA_Grp

61-65

Dur_Grp

21-30

102.4%

AA_Grp

66-70

Dur_Grp

21-30

103.3%

AA_Grp

71-75

Dur_Grp

21-30

101.1%

AA_Grp

81-85

Dur_Grp

21-30

103.8%

AA_Grp

86-90

Dur_Grp

21-30

103.3%

AA_Grp

91-95

Dur_Grp

21-30

104.9%

AA_Grp

61-65

Dur_Grp

31-120

100.3%

AA_Grp

71-75

Dur_Grp

31-120

103.0%

AA_Grp

76-80

Dur_Grp

31-120

103.6%

AA_Grp

81-85

Dur_Grp

31-120

104.8%

AA_Grp

96-100

Dur_Grp

31-120

93.3%

AA_Grp

101-105

Dur_Grp

31-120

75.9%

AA_Grp

106-110

Dur_Grp

31-120

87.8%

AA_Grp

111-115

Dur_Grp

31-120

90.4%

AA_Grp

116-120

Dur_Grp

31-120

98.9%

Sex

M

Face_Amount_Band

10K - <25K

111.3%

Sex

M

Face_Amount_Band

25K - <50K

108.3%

Sex

M

Face_Amount_Band

50K - <100K

107.7%

Sex

M

Face_Amount_Band

100K - <250K

104.5%

Sex

M

Face_Amount_Band

250K - <500K

99.5%

Sex

M

Face_Amount_Band

500K - <1M

97.4%

Sex

M

Face_Amount_Band

1M - <2.5M

91.9%

Sex

M

Face_Amount_Band

2.5M - <5M

93.5%

Sex

M

Face_Amount_Band

5M - <10M

96.6%

Sex

M

Face_Amount_Band

10M+

89.6%

Sex

M

UW

NS/2/2

99.9%

Sex

M

UW

NS/1/3

93.8%

Sex

M

UW

NS/2/3

106.0%

Sex

M

UW

NS/1/4

84.3%

Sex

M

UW

NS/2/4

137.4%

Sex

M

UW

S/1/2

105.8%

Sex

M

UW

S/2/2

94.6%

Sex

M

UW

U/U/U

110.1%

Sex

M

Insurance_Plan

Perm

97.8%

Sex

M

Insurance_Plan

UL

104.3%

Sex

M

Insurance_Plan

VL

100.9%

Sex

M

Insurance_Plan

VLSG

100.5%

Sex

M

AA_Grp

18-25

103.0%

Sex

M

AA_Grp

31-35

105.4%

Sex

M

AA_Grp

36-40

121.1%

Sex

M

AA_Grp

51-55

101.3%

Sex

M

AA_Grp

56-60

111.4%

Sex

M

AA_Grp

61-65

106.1%

Sex

M

AA_Grp

71-75

98.7%

Sex

M

AA_Grp

81-85

93.9%

Sex

M

AA_Grp

86-90

100.5%

Sex

M

AA_Grp

96-100

85.2%

Sex

M

Dur_Grp

2-2

118.4%

Sex

M

Dur_Grp

3-3

100.9%

Sex

M

Dur_Grp

6-10

100.7%

Sex

M

Dur_Grp

11-15

96.3%

Sex

M

Dur_Grp

16-20

99.4%

Sex

M

Dur_Grp

21-30

98.3%

Sex

M

Dur_Grp

31-120

99.5%

Note that interaction terms must be considered together. For example, UW interacts with Face Amount Band, so these should be considered in combination. The slate of UW without a second feature apply to the base level of Face Amount Band (and any other factor with which UW interacts).

Since penalization is roughly equivalent to Bayesian credibility using a combination of Gaussian and Laplace priors, these factors are arguably credible. This is true even when dealing with factors with small effect sizes, such as the standalone factor for Duration 3.

Plots and tables of effects are much more digestible. As it happens, all but Sex are involved in feature interactions. Those factors can be read from the table above.

In the plots and tables that follow, we fix all of the other variables at their middle values when extracting final factors.

We generate some supporting tables: a factor grid for attaching to the experience, and a list of interactions present in the model

Code
dat.perm.knw[,..pred.cols] %>%
  mutate(Face_Amount_Band=fct_drop(Face_Amount_Band)) %>%
  lapply(levels) %>%
  expand.grid() -> 
  dat.perm.knw.grid 

dat.perm.knw.grid %>%
  model.Matrix(
    object=glmnetFormula,
    data=.,
    sparse=T
  ) %>%
  predict(
    cvfit.perm.knw,
    newx=.,
    s="lambda.min"
  ) %>% 
  as.vector() ->
  newCoef

dat.perm.knw.grid %>%
  add_column(
    Factor=exp(newCoef)
  )  %>% 
  setDT() -> 
  dat.perm.knw.grid 

dat.perm.knw.grid %>%
  filter(!(Insurance_Plan == "Perm" & UW == "U/U/U")) %>%
  write.xlsx(
           file="dat.perm.knw.grid.xlsx")

reformatCoefs(cvfit.perm.knw, pred.cols) %>%
  filter(Coef != 0 & !is.na(Feature2Name)) %>% 
  select(Feature1Name,Feature2Name) %>% 
  distinct() %>%
  as.list() %>%
  purrr::list_transpose() ->
  perm.knw.int.list
8.2.6.1.3 Plots of Terms

Below are plots of the 2-way interaction terms, with external factors fixed at their middle values.

Effects tend to be subtle, and the following call out the ones that stand out in the plots. Inspecting the coefficient table will reveal where others are located.

  1. Face Amount Band x Sex: The gradient in mortality with increasing face amount band is steeper than for females.
  2. Sex x UW: preferred differentials seem to vary by Sex.
8.2.6.1.4 Tables of Terms

Below are tables of the 2-way interaction terms, with external factors fixed at their middle values.

Face_Amount_Band

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

<10K

89.8%

120.3%

94.5%

78.1%

135.9%

85.2%

88.6%

114.1%

119.5%

108.5%

84.7%

103.7%

101.4%

105.6%

10K - <25K

110.9%

148.5%

116.8%

96.4%

167.9%

105.2%

109.5%

141.0%

147.6%

144.5%

104.6%

128.1%

125.2%

130.5%

25K - <50K

107.4%

143.8%

113.1%

93.4%

162.6%

101.9%

106.0%

136.5%

142.9%

132.4%

101.3%

124.0%

121.3%

126.3%

50K - <100K

95.7%

127.1%

100.7%

83.2%

144.8%

90.8%

94.4%

121.6%

127.3%

114.6%

90.2%

110.5%

108.0%

121.4%

100K - <250K

84.8%

112.3%

89.2%

73.7%

127.5%

80.4%

83.6%

107.7%

112.7%

99.2%

79.9%

97.8%

95.7%

99.7%

250K - <500K

76.6%

103.1%

80.7%

67.2%

115.8%

72.7%

75.6%

97.4%

101.9%

92.5%

72.3%

88.5%

86.9%

83.4%

500K - <1M

76.5%

105.6%

78.4%

66.5%

118.9%

72.6%

75.5%

97.2%

101.8%

91.7%

72.2%

94.8%

84.4%

88.7%

1M - <2.5M

80.7%

116.6%

70.8%

70.1%

129.1%

76.5%

79.6%

102.5%

107.3%

97.4%

76.1%

92.0%

92.0%

69.8%

2.5M - <5M

71.2%

101.3%

75.0%

59.1%

115.4%

67.6%

53.2%

90.5%

125.6%

85.0%

75.4%

77.9%

82.6%

83.8%

5M - <10M

68.7%

86.9%

72.3%

55.1%

104.0%

65.2%

81.1%

87.3%

91.4%

77.1%

56.9%

90.5%

72.9%

80.8%

10M+

45.9%

64.5%

86.7%

39.9%

79.2%

32.7%

45.3%

58.4%

58.6%

51.5%

43.3%

53.0%

59.3%

123.7%

Dur_Grp

<10K

10K - <25K

25K - <50K

50K - <100K

100K - <250K

250K - <500K

500K - <1M

1M - <2.5M

2.5M - <5M

5M - <10M

10M+

1-1

74.8%

92.4%

89.4%

79.4%

70.6%

63.8%

63.7%

67.2%

48.2%

77.4%

53.3%

2-2

80.1%

98.9%

95.8%

85.0%

75.6%

68.3%

68.2%

59.8%

51.7%

82.8%

57.1%

3-3

68.1%

84.1%

81.5%

72.3%

64.3%

58.1%

58.0%

61.2%

43.9%

70.5%

48.6%

4-5

88.6%

109.5%

106.0%

94.4%

83.6%

75.6%

75.5%

79.6%

53.2%

81.1%

45.3%

6-10

67.2%

83.2%

83.2%

71.3%

62.4%

57.3%

57.3%

57.1%

41.6%

69.5%

60.2%

11-15

66.2%

81.8%

79.2%

70.3%

61.4%

56.5%

56.4%

59.3%

40.6%

69.5%

44.8%

16-20

71.3%

88.1%

84.3%

74.3%

63.7%

60.2%

60.7%

67.5%

47.0%

73.7%

50.8%

21-30

78.8%

90.1%

85.2%

77.1%

69.1%

64.8%

67.2%

72.7%

57.0%

81.5%

57.0%

31-120

69.3%

82.6%

79.0%

73.5%

65.5%

60.8%

59.0%

61.3%

44.7%

71.6%

49.4%

Face_Amount_Band

Other

Perm

UL

ULSG

VL

VLSG

<10K

79.7%

74.3%

88.6%

64.3%

82.5%

75.1%

10K - <25K

98.4%

92.6%

109.5%

79.4%

101.8%

92.8%

25K - <50K

95.1%

88.7%

106.0%

76.8%

98.4%

89.6%

50K - <100K

82.5%

77.0%

94.4%

68.1%

84.4%

77.8%

100K - <250K

73.8%

68.1%

83.6%

59.6%

74.5%

68.2%

250K - <500K

66.7%

57.8%

75.6%

53.8%

67.6%

62.8%

500K - <1M

67.3%

55.8%

75.5%

53.6%

69.7%

61.7%

1M - <2.5M

71.9%

60.8%

79.6%

58.0%

74.4%

67.7%

2.5M - <5M

49.6%

39.7%

53.2%

39.6%

54.7%

51.3%

5M - <10M

78.0%

64.0%

81.1%

63.0%

80.7%

73.5%

10M+

40.7%

71.2%

45.3%

32.1%

42.1%

47.0%

AA_Grp

<10K

10K - <25K

25K - <50K

50K - <100K

100K - <250K

250K - <500K

500K - <1M

1M - <2.5M

2.5M - <5M

5M - <10M

10M+

0-17

98.1%

112.6%

111.6%

101.4%

94.0%

89.4%

89.0%

93.6%

65.9%

89.8%

50.2%

18-25

98.1%

112.6%

111.6%

101.4%

94.0%

89.4%

89.0%

93.6%

65.9%

89.8%

110.9%

26-30

117.7%

135.1%

133.9%

121.6%

112.8%

107.2%

106.8%

112.3%

79.1%

107.7%

60.2%

31-35

111.6%

128.1%

126.9%

115.3%

106.9%

101.6%

101.2%

106.4%

75.0%

102.1%

57.1%

36-40

98.3%

112.8%

111.8%

101.6%

94.2%

88.2%

89.2%

93.8%

66.0%

90.0%

86.2%

41-45

104.2%

119.6%

118.5%

107.7%

99.8%

94.9%

94.5%

99.4%

76.4%

95.3%

53.1%

46-50

108.1%

124.1%

134.0%

111.7%

100.7%

98.0%

96.4%

103.1%

72.6%

98.9%

75.1%

51-55

101.4%

135.6%

132.2%

111.8%

97.2%

92.4%

91.2%

96.8%

62.8%

81.6%

39.2%

56-60

86.7%

104.5%

104.1%

90.0%

81.0%

76.1%

73.3%

74.3%

50.5%

79.4%

44.4%

61-65

88.6%

109.5%

106.0%

94.4%

83.6%

75.6%

75.5%

79.6%

53.2%

81.1%

45.3%

66-70

91.1%

106.8%

104.1%

94.1%

87.3%

80.8%

80.3%

85.9%

61.2%

85.5%

46.6%

71-75

107.9%

123.9%

122.8%

111.5%

104.4%

98.3%

99.8%

101.0%

69.5%

86.0%

41.0%

76-80

99.1%

104.1%

101.8%

96.7%

94.9%

93.6%

92.4%

97.6%

66.5%

90.6%

50.7%

81-85

99.3%

101.3%

103.6%

101.7%

101.7%

102.4%

100.9%

101.3%

66.7%

87.7%

44.0%

86-90

130.4%

115.0%

121.1%

118.8%

125.0%

123.2%

122.7%

124.4%

87.6%

121.5%

79.4%

91-95

79.7%

78.6%

82.6%

81.9%

82.5%

82.1%

81.4%

74.9%

55.6%

72.9%

40.5%

96-100

106.0%

121.6%

120.6%

109.5%

103.9%

96.5%

102.2%

101.1%

71.2%

97.0%

41.5%

101-105

66.9%

76.7%

76.0%

69.1%

64.0%

60.9%

60.6%

63.8%

44.9%

61.2%

34.2%

106-110

61.0%

70.1%

69.4%

63.1%

58.5%

55.6%

55.4%

58.2%

41.0%

55.9%

31.2%

111-115

75.7%

86.9%

86.2%

78.3%

72.6%

69.0%

68.7%

72.2%

50.9%

69.3%

38.7%

116-120

94.2%

108.1%

107.1%

97.3%

90.2%

85.8%

85.5%

89.8%

63.3%

86.2%

48.2%

Insurance_Plan

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

Other

77.8%

102.3%

93.6%

70.0%

117.1%

73.8%

73.8%

98.8%

103.4%

92.6%

73.3%

89.8%

87.8%

91.5%

Perm

71.7%

97.2%

86.3%

70.2%

125.4%

68.0%

68.1%

91.1%

95.4%

85.0%

67.6%

88.5%

83.2%

84.3%

UL

84.8%

112.3%

89.2%

73.7%

127.5%

80.4%

83.6%

107.7%

112.7%

99.2%

79.9%

97.8%

95.7%

99.7%

ULSG

70.2%

92.6%

71.3%

63.2%

105.6%

57.1%

59.6%

91.0%

96.1%

88.3%

66.2%

81.0%

94.5%

66.0%

VL

78.5%

102.8%

94.4%

70.7%

129.9%

74.4%

74.5%

99.7%

104.4%

96.7%

74.0%

90.6%

85.6%

93.2%

VLSG

71.8%

94.5%

86.4%

62.5%

102.4%

68.1%

68.2%

91.2%

95.5%

90.0%

67.7%

82.9%

77.7%

93.4%

Dur_Grp

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

1-1

92.0%

128.6%

86.6%

86.5%

138.4%

87.2%

70.6%

92.0%

116.7%

103.2%

86.7%

113.8%

103.8%

108.2%

2-2

98.5%

137.7%

92.8%

92.7%

148.3%

86.4%

75.6%

98.5%

125.0%

110.5%

92.9%

121.9%

111.2%

115.9%

3-3

89.9%

126.3%

84.7%

84.6%

132.7%

85.3%

64.3%

89.9%

114.1%

105.1%

84.8%

111.3%

101.5%

105.8%

4-5

84.8%

112.3%

89.2%

73.7%

127.5%

80.4%

83.6%

107.7%

112.7%

99.2%

79.9%

97.8%

95.7%

99.7%

6-10

81.3%

112.8%

81.4%

76.5%

127.8%

77.1%

62.4%

81.3%

103.1%

90.3%

68.4%

84.3%

91.8%

115.6%

11-15

80.0%

115.3%

89.0%

72.7%

124.7%

75.9%

61.4%

103.7%

101.5%

87.6%

69.2%

107.9%

88.8%

94.1%

16-20

83.1%

116.8%

74.2%

78.4%

130.1%

78.8%

63.7%

83.1%

105.4%

87.6%

85.3%

107.8%

92.9%

97.7%

21-30

90.1%

126.8%

84.8%

77.1%

60.5%

85.4%

69.1%

90.1%

79.7%

101.0%

90.8%

119.1%

105.8%

106.1%

31-120

85.5%

119.5%

80.5%

80.4%

75.7%

81.1%

65.5%

85.5%

108.4%

95.9%

80.6%

105.7%

96.5%

82.7%

AA_Grp

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

0-17

90.4%

112.7%

82.6%

78.6%

115.5%

90.4%

94.0%

114.9%

111.4%

105.9%

85.2%

104.4%

107.5%

96.2%

18-25

90.4%

112.7%

82.6%

78.6%

115.5%

90.4%

94.0%

114.9%

111.4%

101.1%

85.2%

104.4%

107.5%

96.2%

26-30

108.5%

135.2%

99.1%

94.3%

138.5%

108.5%

112.8%

137.8%

133.7%

127.0%

102.3%

125.2%

129.0%

115.4%

31-35

102.8%

128.2%

87.9%

89.4%

131.3%

102.8%

106.9%

130.6%

126.7%

120.4%

96.9%

118.7%

122.3%

109.4%

36-40

90.6%

112.9%

82.8%

78.7%

126.0%

90.6%

94.2%

115.1%

111.6%

106.1%

85.4%

104.6%

107.7%

96.4%

41-45

96.0%

119.7%

89.5%

85.1%

122.7%

96.0%

99.8%

122.0%

118.3%

112.4%

86.2%

110.8%

114.2%

102.1%

46-50

96.8%

128.5%

83.2%

84.1%

168.6%

96.8%

100.7%

123.0%

119.3%

114.1%

91.3%

111.8%

114.0%

103.0%

51-55

93.5%

116.5%

85.4%

81.2%

133.6%

93.5%

97.2%

118.7%

115.2%

107.5%

88.1%

107.9%

106.3%

99.4%

56-60

77.9%

97.1%

79.8%

67.7%

99.5%

77.9%

81.0%

98.9%

96.0%

90.3%

73.4%

97.5%

92.6%

91.6%

61-65

84.8%

112.3%

89.2%

73.7%

127.5%

80.4%

83.6%

107.7%

112.7%

99.2%

79.9%

97.8%

95.7%

99.7%

66-70

84.0%

115.3%

60.3%

71.2%

114.4%

80.6%

87.3%

106.8%

103.6%

111.6%

79.2%

97.0%

101.8%

118.8%

71-75

100.4%

125.2%

103.8%

87.3%

134.1%

100.4%

104.4%

127.6%

142.1%

118.3%

98.8%

115.8%

119.4%

115.4%

76-80

96.2%

120.0%

82.9%

72.0%

116.1%

96.2%

94.9%

138.2%

118.6%

112.7%

109.5%

111.1%

122.9%

106.2%

81-85

97.8%

117.5%

90.4%

82.9%

117.5%

97.8%

101.7%

125.9%

120.5%

114.5%

92.2%

116.1%

129.0%

104.0%

86-90

96.3%

120.1%

74.4%

74.7%

101.0%

96.3%

125.0%

91.5%

106.4%

118.0%

90.8%

111.2%

118.3%

102.5%

91-95

79.4%

88.5%

65.7%

73.1%

79.0%

79.4%

82.5%

100.9%

85.2%

95.0%

74.8%

91.6%

80.9%

84.4%

96-100

99.9%

122.4%

91.3%

89.8%

104.0%

99.9%

103.9%

126.9%

123.1%

117.0%

94.2%

115.3%

118.8%

104.9%

101-105

61.6%

76.8%

56.3%

53.5%

78.7%

61.6%

64.0%

78.3%

75.9%

72.1%

58.1%

71.1%

73.3%

61.6%

106-110

56.2%

70.1%

51.4%

48.9%

71.8%

56.2%

58.5%

71.5%

69.3%

65.9%

53.0%

64.9%

66.9%

59.8%

111-115

69.8%

87.0%

63.8%

60.7%

89.1%

69.8%

72.6%

88.7%

86.0%

81.7%

65.8%

80.6%

83.0%

74.2%

116-120

86.8%

108.2%

79.3%

75.4%

110.9%

86.8%

90.2%

110.3%

107.0%

101.6%

81.8%

100.2%

103.2%

92.3%

Dur_Grp

Other

Perm

UL

ULSG

VL

VLSG

1-1

62.3%

57.4%

70.6%

51.2%

62.8%

59.0%

2-2

66.7%

73.7%

75.6%

49.4%

67.3%

63.2%

3-3

56.7%

52.3%

64.3%

46.7%

57.2%

53.8%

4-5

73.8%

68.1%

83.6%

59.6%

74.5%

68.2%

6-10

59.0%

54.4%

62.4%

49.0%

59.5%

54.9%

11-15

59.8%

55.1%

61.4%

48.6%

60.3%

62.6%

16-20

58.4%

52.9%

63.7%

49.9%

58.9%

55.3%

21-30

59.2%

51.7%

69.1%

48.7%

59.8%

56.1%

31-120

57.9%

53.3%

65.5%

29.8%

58.4%

54.8%

AA_Grp

Other

Perm

UL

ULSG

VL

VLSG

0-17

83.0%

82.0%

94.0%

72.9%

80.9%

76.6%

18-25

83.0%

82.0%

94.0%

72.9%

81.5%

76.6%

26-30

99.5%

98.4%

112.8%

87.4%

97.1%

91.9%

31-35

94.4%

93.3%

106.9%

82.9%

92.0%

87.1%

36-40

83.1%

82.2%

94.2%

73.0%

81.1%

76.8%

41-45

88.1%

87.1%

99.8%

80.8%

85.9%

81.4%

46-50

82.6%

81.7%

100.7%

72.5%

73.8%

76.3%

51-55

83.0%

81.6%

97.2%

82.5%

80.9%

75.8%

56-60

69.1%

68.3%

81.0%

60.7%

66.5%

63.8%

61-65

73.8%

68.1%

83.6%

59.6%

74.5%

68.2%

66-70

82.9%

71.4%

87.3%

71.3%

80.8%

76.5%

71-75

89.3%

82.4%

104.4%

72.4%

90.9%

85.1%

76-80

80.4%

79.5%

94.9%

70.6%

75.7%

74.1%

81-85

88.0%

87.5%

101.7%

76.8%

85.8%

81.2%

86-90

113.7%

115.3%

125.0%

100.8%

113.2%

105.0%

91-95

78.9%

79.4%

82.5%

70.3%

74.6%

72.8%

96-100

91.7%

94.3%

103.9%

68.5%

97.3%

84.6%

101-105

68.6%

67.8%

64.0%

60.2%

66.9%

63.3%

106-110

51.6%

51.0%

58.5%

45.3%

50.3%

47.7%

111-115

64.0%

63.3%

72.6%

56.2%

62.5%

59.1%

116-120

79.6%

78.7%

90.2%

69.9%

77.7%

73.5%

AA_Grp

1-1

2-2

3-3

4-5

6-10

11-15

16-20

21-30

31-120

0-17

77.1%

77.1%

71.8%

94.0%

68.1%

67.2%

69.6%

73.7%

71.4%

18-25

77.1%

77.1%

71.8%

94.0%

76.9%

67.2%

69.6%

73.7%

71.4%

26-30

92.5%

92.5%

86.2%

112.8%

81.7%

80.7%

83.5%

88.4%

85.7%

31-35

87.7%

87.7%

81.7%

106.9%

77.5%

76.5%

79.2%

83.9%

81.2%

36-40

77.2%

77.2%

72.0%

94.2%

68.3%

67.4%

69.7%

73.9%

71.6%

41-45

81.9%

82.2%

82.2%

99.8%

72.4%

71.4%

73.9%

75.6%

75.9%

46-50

83.7%

76.9%

78.0%

100.7%

82.5%

75.9%

75.6%

79.9%

77.6%

51-55

79.7%

62.2%

74.2%

97.2%

70.1%

69.5%

72.0%

76.2%

73.8%

56-60

74.2%

74.2%

66.8%

81.0%

73.5%

64.7%

66.8%

70.8%

68.8%

61-65

70.6%

75.6%

64.3%

83.6%

62.4%

61.4%

63.7%

69.1%

65.5%

66-70

71.6%

58.6%

66.7%

87.3%

61.0%

61.3%

66.6%

70.7%

66.3%

71-75

80.8%

80.8%

74.4%

104.4%

73.1%

69.8%

73.0%

78.2%

77.1%

76-80

77.8%

77.8%

72.5%

94.9%

63.1%

73.1%

69.4%

74.4%

74.7%

81-85

84.1%

84.1%

78.3%

101.7%

69.9%

73.3%

75.8%

83.5%

81.7%

86-90

96.8%

96.8%

90.2%

125.0%

84.8%

77.2%

89.9%

95.7%

89.7%

91-95

77.0%

77.0%

71.8%

82.5%

52.9%

66.4%

75.4%

77.3%

71.4%

96-100

85.2%

85.2%

79.4%

103.9%

90.2%

77.4%

76.9%

81.5%

73.7%

101-105

52.5%

52.5%

48.9%

64.0%

46.4%

45.8%

52.4%

50.2%

37.0%

106-110

48.0%

48.0%

44.7%

58.5%

42.4%

41.8%

43.3%

45.9%

39.0%

111-115

59.5%

59.5%

55.4%

72.6%

52.6%

51.9%

53.7%

56.9%

49.9%

116-120

74.0%

74.0%

68.9%

90.2%

65.4%

64.5%

66.8%

70.8%

67.8%

Face_Amount_Band

F

M

<10K

88.6%

137.0%

10K - <25K

109.5%

188.4%

25K - <50K

106.0%

177.5%

50K - <100K

94.4%

157.3%

100K - <250K

83.6%

135.1%

250K - <500K

75.6%

116.3%

500K - <1M

75.5%

113.8%

1M - <2.5M

79.6%

113.1%

2.5M - <5M

53.2%

76.9%

5M - <10M

81.1%

121.1%

10M+

45.3%

62.8%

Sex

NS/1/2

NS/2/2

NS/1/3

NS/2/3

NS/3/3

NS/1/4

NS/2/4

NS/3/4

NS/4/4

NS/U/U

S/1/2

S/2/2

S/U/U

U/U/U

F

84.8%

112.3%

89.2%

73.7%

127.5%

80.4%

83.6%

107.7%

112.7%

99.2%

79.9%

97.8%

95.7%

99.7%

M

99.6%

131.8%

98.3%

91.8%

149.9%

79.6%

135.1%

126.6%

132.5%

116.7%

99.4%

108.8%

112.5%

129.0%

Insurance_Plan

F

M

Other

73.8%

114.3%

Perm

68.1%

103.1%

UL

83.6%

135.1%

ULSG

59.6%

92.3%

VL

74.5%

116.4%

VLSG

68.2%

106.0%

AA_Grp

F

M

0-17

94.0%

143.1%

18-25

94.0%

147.3%

26-30

112.8%

171.6%

31-35

106.9%

171.5%

36-40

94.2%

173.6%

41-45

99.8%

151.9%

46-50

100.7%

153.2%

51-55

97.2%

149.8%

56-60

81.0%

137.3%

61-65

83.6%

135.1%

66-70

87.3%

132.8%

71-75

104.4%

156.9%

76-80

94.9%

144.4%

81-85

101.7%

145.3%

86-90

125.0%

191.1%

91-95

82.5%

125.6%

96-100

103.9%

134.6%

101-105

64.0%

97.5%

106-110

58.5%

89.0%

111-115

72.6%

110.4%

116-120

90.2%

137.3%

Dur_Grp

F

M

1-1

70.6%

114.0%

2-2

75.6%

144.5%

3-3

64.3%

104.7%

4-5

83.6%

135.1%

6-10

62.4%

101.4%

11-15

61.4%

95.4%

16-20

63.7%

102.3%

21-30

69.1%

109.7%

31-120

65.5%

105.3%

8.2.6.2 Goodness-of-Fit

Next comes checking goodness-of-fit, on both a univariate and bivariate basis. Tables are provided which show the ratio of the actual to model predicted deaths and show the associated death counts.

Code
dat.perm.knw.grid %>%
  right_join(dat.perm.knw) %>%
  mutate(ExpDth_Amt_GLMNet=ExpDth_Amt_VBT2015*Factor) %>%
  mutate(Factor=NULL) ->
  dat.perm.knw
8.2.6.2.1 Univariate Fit Checks

Sex

Death Count

Actual-to-Model

F

509,660

100.0%

M

693,462

100.0%

Face_Amount_Band

Death Count

Actual-to-Model

<10K

144,194

113.4%

10K - <25K

284,132

100.5%

25K - <50K

244,005

100.2%

50K - <100K

246,768

100.1%

100K - <250K

194,495

99.9%

250K - <500K

46,489

99.8%

500K - <1M

22,485

99.8%

1M - <2.5M

14,772

99.9%

2.5M - <5M

2,919

99.8%

5M - <10M

2,025

99.8%

10M+

838

99.8%

Insurance_Plan

Death Count

Actual-to-Model

Other

2,986

98.5%

Perm

628,166

100.0%

UL

406,569

100.0%

ULSG

72,864

100.0%

VL

72,371

100.1%

VLSG

20,166

99.6%

UW

Death Count

Actual-to-Model

NS/1/2

45,097

99.6%

NS/2/2

84,588

100.1%

NS/1/3

7,038

99.5%

NS/2/3

13,974

99.8%

NS/3/3

35,709

100.1%

NS/1/4

2,397

98.4%

NS/2/4

2,766

98.3%

NS/3/4

2,035

101.8%

NS/4/4

4,180

101.2%

NS/U/U

710,163

100.0%

S/1/2

8,012

98.7%

S/2/2

14,701

101.1%

S/U/U

241,685

100.2%

U/U/U

30,777

101.0%

AA_Grp

Death Count

Actual-to-Model

0-17

1,588

73.2%

18-25

6,083

101.8%

26-30

5,395

105.7%

31-35

6,362

103.8%

36-40

8,206

101.8%

41-45

13,458

101.1%

46-50

24,192

100.6%

51-55

43,169

99.8%

56-60

66,765

99.8%

61-65

95,105

99.8%

66-70

122,641

99.9%

71-75

146,304

100.1%

76-80

173,224

100.1%

81-85

203,208

100.0%

86-90

183,307

100.1%

91-95

87,840

99.9%

96-100

15,512

100.7%

101-105

664

87.7%

106-110

55

22.0%

111-115

31

7.4%

116-120

13

4.5%

Dur_Grp

Death Count

Actual-to-Model

1-1

5,348

108.8%

2-2

6,945

99.9%

3-3

8,476

100.9%

4-5

19,807

99.6%

6-10

66,221

99.9%

11-15

101,761

99.9%

16-20

170,607

99.9%

21-30

670,560

100.0%

31-120

153,397

99.8%

Unidimensional goodness-of-fit is generally good, with some spots with poor fit. Examples of poor fit include the very lowest face amounts, the extremes of attained age (juveniles and the oldest ages), and duration 1.

8.2.6.2.2 Bivariate Fit Checks

Face_Amount_Band

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

<10K

95,801

107.5%

48,393

126.6%

10K - <25K

152,406

100.0%

131,726

100.9%

25K - <50K

94,543

100.0%

149,462

100.4%

50K - <100K

79,467

99.9%

167,301

100.2%

100K - <250K

56,221

99.5%

138,274

100.1%

250K - <500K

14,847

99.9%

31,642

99.8%

500K - <1M

7,842

99.8%

14,643

99.8%

1M - <2.5M

5,976

100.0%

8,796

99.8%

2.5M - <5M

1,314

100.0%

1,605

99.7%

5M - <10M

865

100.0%

1,160

99.7%

10M+

378

99.9%

460

99.7%

Insurance_Plan

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

Other

949

105.1%

2,037

95.4%

Perm

295,165

100.2%

333,001

99.9%

UL

150,335

100.0%

256,234

100.1%

ULSG

30,909

100.0%

41,955

100.0%

VL

24,150

99.7%

48,221

100.2%

VLSG

8,152

97.5%

12,014

100.6%

UW

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

NS/1/2

15,578

98.9%

29,519

100.0%

NS/2/2

34,658

100.5%

49,930

99.8%

NS/1/3

2,955

99.9%

4,083

99.3%

NS/2/3

5,515

99.1%

8,459

100.3%

NS/3/3

14,796

100.1%

20,913

100.1%

NS/1/4

1,074

100.9%

1,323

96.4%

NS/2/4

1,064

90.6%

1,702

102.8%

NS/3/4

718

103.0%

1,317

101.2%

NS/4/4

1,752

103.5%

2,428

99.8%

NS/U/U

306,182

100.2%

403,981

100.0%

S/1/2

2,526

93.0%

5,486

101.9%

S/2/2

5,574

105.3%

9,127

98.4%

S/U/U

107,653

100.5%

134,032

100.0%

U/U/U

9,615

99.8%

21,162

101.4%

AA_Grp

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

0-17

577

70.3%

1,011

74.7%

18-25

1,592

89.5%

4,491

105.6%

26-30

1,515

106.1%

3,880

105.5%

31-35

2,090

100.8%

4,272

105.2%

36-40

2,926

98.9%

5,280

103.1%

41-45

5,089

101.7%

8,369

100.8%

46-50

8,920

101.2%

15,272

100.4%

51-55

15,811

98.1%

27,358

100.5%

56-60

23,303

98.2%

43,462

100.3%

61-65

33,547

98.7%

61,558

100.2%

66-70

43,958

100.0%

78,683

99.9%

71-75

55,088

101.0%

91,216

99.8%

76-80

69,560

100.4%

103,664

100.0%

81-85

91,310

100.2%

111,898

99.9%

86-90

92,578

100.0%

90,729

100.1%

91-95

51,021

99.8%

36,819

100.1%

96-100

10,253

102.1%

5,259

97.4%

101-105

475

94.9%

189

81.6%

106-110

31

30.2%

24

17.2%

111-115

13

11.3%

18

6.0%

116-120

3

3.4%

10

4.9%

Dur_Grp

Sex: F

Sex: M

Deaths

Ratio

Deaths

Ratio

1-1

2,066

115.9%

3,282

105.6%

2-2

2,797

94.6%

4,148

102.4%

3-3

3,602

99.6%

4,874

101.7%

4-5

8,694

98.7%

11,113

100.2%

6-10

29,892

99.7%

36,329

100.1%

11-15

47,079

100.0%

54,682

99.9%

16-20

80,747

100.0%

89,860

99.8%

21-30

281,086

100.2%

389,474

99.9%

31-120

53,697

99.9%

99,700

99.8%

Face_Amount_Band

Insurance_Plan: Other

Insurance_Plan: Perm

Insurance_Plan: UL

Insurance_Plan: ULSG

Insurance_Plan: VL

Insurance_Plan: VLSG

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

<10K

809

120.7%

131,213

113.1%

9,445

114.5%

159

104.3%

942

131.9%

1,626

122.1%

10K - <25K

873

113.5%

220,099

100.5%

57,377

99.1%

1,861

107.4%

1,968

120.5%

1,954

101.2%

25K - <50K

438

100.7%

116,434

99.8%

106,894

100.5%

7,988

103.6%

8,502

102.1%

3,749

97.5%

50K - <100K

291

103.9%

77,321

100.0%

119,421

100.2%

15,049

102.1%

31,874

99.1%

2,812

97.3%

100K - <250K

377

115.0%

58,883

99.8%

85,089

100.1%

24,815

100.3%

19,722

99.3%

5,609

97.7%

250K - <500K

118

126.5%

14,052

99.6%

15,258

100.4%

9,322

100.1%

5,236

99.0%

2,503

98.3%

500K - <1M

35

90.3%

6,276

99.5%

6,655

100.4%

5,914

99.5%

2,473

100.9%

1,132

97.5%

1M - <2.5M

31

137.5%

3,271

99.6%

4,256

99.7%

5,217

99.8%

1,355

100.9%

642

100.7%

2.5M - <5M

7

107.6%

433

98.6%

963

99.4%

1,224

99.6%

198

102.8%

94

106.3%

5M - <10M

5

95.4%

129

97.7%

834

99.6%

957

100.3%

69

103.5%

31

95.3%

10M+

2

41.9%

55

103.2%

377

99.8%

358

99.7%

32

96.4%

14

110.1%

UW

Face_Amount_Band: <10K

Face_Amount_Band: 10K - <25K

Face_Amount_Band: 25K - <50K

Face_Amount_Band: 50K - <100K

Face_Amount_Band: 100K - <250K

Face_Amount_Band: 250K - <500K

Face_Amount_Band: 500K - <1M

Face_Amount_Band: 1M - <2

Face_Amount_Band: 2

Face_Amount_Band: 5M - <10M

Face_Amount_Band: 10M+

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

5M

5M - <5M

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

NS/1/2

104

185.3%

805

119.5%

3,338

107.7%

7,353

109.0%

20,096

99.5%

6,769

99.5%

3,522

100.2%

2,403

98.6%

383

96.0%

248

103.2%

76

96.3%

NS/2/2

4,345

106.2%

13,900

97.8%

15,618

97.1%

17,769

98.5%

19,846

99.4%

6,421

100.8%

3,453

100.8%

2,387

100.6%

466

101.2%

294

99.0%

89

101.6%

NS/1/3

3

32.8%

16

151.0%

128

109.1%

701

101.1%

2,807

100.1%

1,542

97.7%

906

96.7%

614

97.6%

151

102.7%

92

97.9%

78

102.7%

NS/2/3

7

225.0%

238

151.9%

647

112.6%

1,647

108.5%

5,064

101.6%

2,507

102.2%

1,573

99.5%

1,443

100.1%

365

98.5%

336

99.1%

147

99.1%

NS/3/3

337

132.6%

1,919

100.5%

3,707

104.3%

5,760

96.7%

12,232

99.0%

4,730

98.9%

2,919

101.0%

2,447

100.6%

698

100.8%

666

99.7%

294

100.6%

NS/1/4

0

0.0%

2

122.9%

136

95.4%

229

104.6%

993

99.1%

503

105.2%

301

106.1%

176

99.5%

31

86.6%

21

104.7%

5

78.1%

NS/2/4

3

920.9%

6

277.5%

271

106.5%

363

100.8%

1,130

102.4%

428

96.4%

271

99.1%

213

97.1%

30

82.2%

37

109.6%

14

94.9%

NS/3/4

0

0.0%

0

0.0%

134

96.6%

189

95.3%

973

101.6%

334

101.0%

173

94.4%

162

103.8%

38

113.1%

22

93.5%

10

105.2%

NS/4/4

6

245.9%

126

98.6%

580

91.9%

900

97.2%

1,409

103.2%

482

95.9%

298

102.9%

244

102.2%

78

109.2%

43

102.9%

14

90.7%

NS/U/U

93,900

118.0%

180,839

100.7%

154,030

100.4%

153,961

99.8%

98,451

99.9%

17,596

99.8%

6,922

99.6%

3,642

100.3%

525

99.0%

216

98.6%

81

98.4%

S/1/2

43

145.5%

510

117.9%

954

99.7%

1,705

103.2%

3,606

101.6%

703

95.5%

268

91.3%

175

105.0%

35

118.8%

7

69.3%

6

80.6%

S/2/2

472

108.1%

2,681

102.9%

3,245

102.3%

3,573

101.6%

3,353

102.9%

727

102.8%

386

108.3%

205

93.7%

28

83.5%

21

117.7%

10

94.3%

S/U/U

42,091

103.9%

76,218

100.4%

52,987

99.0%

44,425

99.6%

20,796

99.9%

3,164

101.7%

1,300

97.9%

594

102.2%

81

107.0%

19

86.4%

10

115.6%

U/U/U

2,883

121.0%

6,872

95.5%

8,230

106.5%

8,193

103.9%

3,739

101.0%

583

90.8%

193

87.6%

67

78.7%

10

102.2%

3

154.2%

4

228.9%

AA_Grp

Face_Amount_Band: <10K

Face_Amount_Band: 10K - <25K

Face_Amount_Band: 25K - <50K

Face_Amount_Band: 50K - <100K

Face_Amount_Band: 100K - <250K

Face_Amount_Band: 250K - <500K

Face_Amount_Band: 500K - <1M

Face_Amount_Band: 1M - <2

Face_Amount_Band: 2

Face_Amount_Band: 5M - <10M

Face_Amount_Band: 10M+

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

5M

5M - <5M

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

0-17

25

192.9%

218

75.5%

550

80.9%

504

74.2%

241

79.8%

36

61.8%

10

59.1%

3

58.7%

1

296.2%

0

0.0%

0

0.0%

18-25

139

138.9%

1,312

95.5%

1,816

93.5%

1,787

95.0%

832

96.1%

130

80.4%

41

76.8%

19

119.4%

2

152.9%

1

100.0%

4

237.2%

26-30

206

148.4%

1,221

100.2%

1,545

107.0%

1,420

112.0%

758

106.6%

156

94.9%

60

108.8%

24

125.3%

5

270.0%

0

0.0%

0

0.0%

31-35

230

172.6%

1,296

104.3%

1,749

112.3%

1,587

114.6%

1,040

111.3%

296

107.1%

110

95.0%

49

105.4%

5

138.3%

0

0.0%

0

0.0%

36-40

285

176.5%

1,465

105.8%

2,079

105.6%

1,974

101.2%

1,562

94.6%

478

88.7%

217

88.5%

122

112.1%

13

127.0%

5

113.0%

6

160.4%

41-45

547

211.5%

2,206

129.3%

3,022

118.4%

3,125

100.9%

2,964

96.2%

915

99.3%

427

99.2%

211

98.7%

29

123.9%

9

130.9%

3

65.3%

46-50

1,025

244.9%

3,536

143.2%

5,023

113.3%

6,157

103.0%

5,848

97.8%

1,525

96.4%

654

95.7%

359

99.1%

38

95.6%

17

110.7%

10

117.1%

51-55

1,460

229.5%

5,886

129.4%

8,491

107.6%

12,148

102.5%

11,082

100.1%

2,464

99.5%

1,018

97.2%

541

101.0%

54

90.3%

18

84.1%

7

81.3%

56-60

2,006

171.5%

8,820

115.7%

13,371

104.3%

19,303

101.5%

17,347

99.2%

3,614

98.4%

1,451

97.9%

716

98.0%

79

93.1%

42

103.2%

16

104.4%

61-65

3,262

153.5%

14,805

109.0%

19,969

102.8%

27,605

101.1%

22,195

99.4%

4,325

98.7%

1,845

98.4%

922

98.5%

104

94.9%

52

103.0%

21

101.6%

66-70

5,970

138.2%

24,155

105.1%

27,700

102.0%

32,505

100.6%

24,177

99.7%

4,718

98.8%

2,046

98.6%

1,118

98.8%

151

97.4%

73

104.3%

28

98.3%

71-75

11,618

135.6%

35,392

103.2%

33,679

100.0%

33,402

100.6%

23,492

100.5%

4,855

100.7%

2,271

101.3%

1,281

99.0%

199

97.3%

82

96.3%

33

94.9%

76-80

21,901

118.2%

46,689

97.4%

37,370

98.4%

32,761

99.1%

24,068

100.1%

5,479

101.0%

2,601

101.1%

1,731

100.8%

334

101.6%

198

98.8%

92

99.5%

81-85

35,530

111.9%

57,081

97.8%

39,491

98.5%

32,457

99.2%

24,964

100.5%

6,545

100.9%

3,399

100.9%

2,535

100.5%

571

99.9%

437

99.3%

198

99.2%

86-90

37,686

98.3%

51,750

97.2%

31,811

98.0%

25,835

98.9%

21,428

100.1%

6,396

100.8%

3,658

100.8%

2,957

100.4%

795

99.6%

686

100.4%

305

100.5%

91-95

18,828

109.3%

24,228

94.8%

13,911

96.2%

11,988

97.9%

10,368

101.2%

3,709

101.5%

2,182

101.4%

1,731

99.3%

445

101.2%

350

100.0%

100

98.7%

96-100

3,370

103.9%

3,891

83.5%

2,305

88.1%

2,078

96.9%

2,001

106.0%

805

105.9%

477

106.1%

429

101.3%

89

102.8%

54

100.2%

13

88.5%

101-105

104

124.4%

155

77.5%

104

73.4%

103

71.7%

113

76.0%

39

86.2%

16

56.3%

22

84.7%

5

106.5%

1

36.4%

2

186.1%

106-110

2

45.0%

13

21.3%

9

19.1%

15

27.8%

9

16.3%

3

14.5%

2

26.8%

2

31.8%

0

0.0%

0

0.0%

111-115

0

0.0%

9

16.7%

7

16.5%

10

27.8%

5

11.9%

0

0.0%

0

0.0%

0

0.0%

0

0.0%

116-120

0

0.0%

4

19.7%

3

15.4%

4

12.7%

1

1.8%

1

5.9%

0

0.0%

0

0.0%

Face_Amount_Band

Dur_Grp: 1-1

Dur_Grp: 2-2

Dur_Grp: 3-3

Dur_Grp: 4-5

Dur_Grp: 6-10

Dur_Grp: 11-15

Dur_Grp: 16-20

Dur_Grp: 21-30

Dur_Grp: 31-120

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

<10K

189

330.5%

273

230.5%

352

220.6%

951

187.5%

2,516

141.2%

7,446

132.2%

24,926

120.5%

89,688

109.3%

17,853

113.1%

10K - <25K

1,056

253.3%

1,229

155.5%

1,387

150.9%

2,959

131.3%

8,277

116.6%

16,386

106.1%

38,473

98.8%

176,131

99.3%

38,234

97.0%

25K - <50K

1,152

154.5%

1,402

122.4%

1,682

124.7%

3,579

112.4%

10,224

106.0%

15,681

103.5%

30,499

98.2%

142,628

99.6%

37,158

98.5%

50K - <100K

1,000

135.6%

1,348

114.2%

1,607

112.0%

3,822

108.0%

11,946

100.7%

18,207

100.2%

31,509

99.1%

142,244

99.8%

35,085

99.5%

100K - <250K

1,147

111.3%

1,646

103.8%

2,007

101.2%

4,925

101.1%

17,426

99.3%

24,729

99.5%

29,849

99.6%

92,538

99.9%

20,228

100.6%

250K - <500K

419

110.6%

553

97.0%

705

100.8%

1,693

99.6%

6,378

99.5%

8,612

100.0%

8,182

99.3%

16,700

99.7%

3,247

101.7%

500K - <1M

215

108.9%

283

94.5%

387

103.0%

917

98.1%

3,792

99.4%

4,946

99.7%

4,079

100.0%

6,768

99.8%

1,098

101.3%

1M - <2.5M

129

98.7%

151

90.3%

264

103.7%

678

100.6%

3,261

99.6%

3,978

99.7%

2,568

100.5%

3,332

100.4%

411

97.1%

2.5M - <5M

23

112.8%

35

96.5%

44

92.6%

150

96.4%

1,006

99.5%

854

99.4%

348

101.6%

397

101.5%

62

103.6%

5M - <10M

12

96.0%

18

108.0%

29

94.1%

98

96.9%

940

99.8%

677

100.4%

131

98.4%

103

102.0%

17

95.4%

10M+

6

92.5%

7

98.1%

12

90.8%

35

95.0%

455

100.4%

245

99.4%

43

101.8%

31

103.6%

4

80.1%

UW

Insurance_Plan: Other

Insurance_Plan: Perm

Insurance_Plan: UL

Insurance_Plan: ULSG

Insurance_Plan: VL

Insurance_Plan: VLSG

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

NS/1/2

28

167.8%

9,854

98.2%

19,946

99.9%

6,492

99.3%

6,729

99.4%

2,048

103.1%

NS/2/2

109

80.5%

29,357

100.5%

35,033

100.3%

7,951

100.4%

9,150

99.3%

2,988

98.4%

NS/1/3

54

262.5%

940

99.4%

2,030

98.5%

3,278

99.0%

376

105.8%

360

94.0%

NS/2/3

54

104.3%

1,214

104.4%

3,880

99.4%

7,607

100.1%

630

96.9%

589

94.2%

NS/3/3

103

102.1%

4,770

102.4%

10,193

100.0%

18,371

100.0%

1,305

103.4%

967

96.1%

NS/1/4

3

22.7%

40

90.5%

167

104.3%

1,893

97.3%

78

83.1%

216

116.6%

NS/2/4

2

70.0%

58

95.4%

281

110.4%

2,202

98.0%

52

108.4%

171

77.3%

NS/3/4

2

1.5%

34

110.0%

207

103.8%

1,503

102.8%

84

109.8%

205

92.5%

NS/4/4

4

17.8%

213

100.8%

889

102.3%

2,747

101.8%

143

113.7%

184

103.7%

NS/U/U

630

107.8%

413,063

99.9%

240,127

99.9%

13,494

100.7%

34,748

100.5%

8,101

102.4%

S/1/2

16

88.9%

2,256

98.7%

2,472

101.8%

1,954

98.5%

916

91.8%

398

101.0%

S/2/2

23

30.5%

3,891

105.4%

6,007

100.3%

2,503

98.9%

1,694

102.6%

583

104.3%

S/U/U

101

132.5%

162,476

100.3%

59,630

100.4%

2,590

107.2%

13,843

98.6%

3,045

91.3%

U/U/U

1,857

103.0%

25,707

100.1%

279

57.9%

2,623

109.5%

311

133.0%

AA_Grp

Insurance_Plan: Other

Insurance_Plan: Perm

Insurance_Plan: UL

Insurance_Plan: ULSG

Insurance_Plan: VL

Insurance_Plan: VLSG

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

0-17

1

18.4%

521

72.7%

699

75.4%

164

72.9%

143

77.3%

60

61.2%

18-25

9

80.8%

2,420

91.1%

2,608

97.8%

326

82.3%

551

126.4%

169

142.9%

26-30

6

76.2%

2,468

112.6%

2,089

100.3%

245

103.5%

474

107.1%

113

94.7%

31-35

8

28.2%

2,943

101.8%

2,412

103.2%

326

118.0%

503

103.1%

170

99.5%

36-40

17

45.0%

3,610

102.2%

3,128

103.6%

442

108.0%

732

104.9%

277

85.7%

41-45

28

146.3%

6,093

98.0%

4,732

103.7%

783

108.1%

1,296

96.4%

526

101.0%

46-50

51

58.2%

11,203

100.7%

8,215

102.1%

1,359

103.1%

2,529

96.4%

835

102.6%

51-55

92

118.5%

19,304

98.9%

15,387

101.2%

2,377

102.9%

4,698

98.0%

1,311

95.4%

56-60

149

77.6%

28,744

100.0%

24,995

100.7%

4,176

99.4%

6,870

98.5%

1,831

99.2%

61-65

186

73.1%

40,716

99.4%

36,575

100.0%

6,393

98.9%

8,972

101.3%

2,263

102.5%

66-70

239

150.0%

55,704

99.5%

45,924

99.6%

8,704

99.2%

9,542

101.1%

2,528

103.2%

71-75

285

102.8%

71,766

99.5%

52,740

100.4%

9,722

99.3%

9,205

101.4%

2,586

104.5%

76-80

408

151.3%

93,678

100.3%

57,803

100.3%

10,492

100.3%

8,388

98.5%

2,455

94.9%

81-85

494

96.1%

117,775

100.3%

62,957

100.2%

11,325

99.7%

8,197

99.4%

2,460

98.0%

86-90

560

68.2%

109,701

100.3%

54,624

99.8%

10,035

100.2%

6,568

101.5%

1,819

97.4%

91-95

342

95.2%

51,937

100.6%

26,775

99.7%

4,941

100.4%

3,140

96.4%

705

94.3%

96-100

104

162.2%

9,255

103.0%

4,626

100.6%

932

97.4%

538

115.9%

57

71.4%

101-105

7

68.9%

320

95.7%

236

63.5%

75

100.6%

25

140.1%

1

132.4%

106-110

0

0.0%

8

204.5%

24

16.4%

23

20.8%

0

0.0%

111-115

0

0.0%

16

7.0%

15

7.9%

116-120

4

1.6%

9

8.9%

Dur_Grp

Insurance_Plan: Other

Insurance_Plan: Perm

Insurance_Plan: UL

Insurance_Plan: ULSG

Insurance_Plan: VL

Insurance_Plan: VLSG

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

1-1

28

62.0%

2,537

127.8%

1,269

121.3%

1,386

103.2%

54

87.7%

74

58.9%

2-2

17

82.3%

2,754

107.4%

1,749

105.8%

2,245

96.6%

65

65.1%

115

95.5%

3-3

24

56.2%

3,097

97.9%

2,198

101.1%

2,861

101.6%

139

93.7%

157

114.3%

4-5

86

117.9%

6,749

101.3%

5,071

100.6%

6,975

99.3%

415

97.4%

511

92.3%

6-10

272

102.0%

17,121

99.9%

19,345

99.8%

24,167

100.1%

2,936

99.8%

2,380

98.1%

11-15

117

79.2%

32,893

99.4%

31,282

99.8%

20,581

99.8%

11,005

100.5%

5,883

101.2%

16-20

67

321.3%

81,452

99.7%

58,148

99.8%

7,129

100.8%

17,580

99.9%

6,231

98.7%

21-30

627

71.5%

384,784

99.9%

240,103

100.1%

6,697

100.6%

34,226

100.3%

4,123

106.6%

31-120

1,748

112.1%

96,779

99.9%

47,404

99.9%

823

71.8%

5,951

101.7%

692

115.5%

AA_Grp

UW: NS/U/U

UW: S/U/U

UW: U/U/U

UW: NS/1/2

UW: NS/2/2

UW: NS/1/3

UW: NS/2/3

UW: NS/3/3

UW: NS/1/4

UW: NS/2/4

UW: NS/3/4

UW: NS/4/4

UW: S/1/2

UW: S/2/2

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

0-17

642

66.1%

405

86.9%

541

75.4%

18-25

2,909

88.8%

1,189

126.6%

1,492

109.2%

73

91.1%

36

97.7%

63

98.0%

30

95.5%

114

67.3%

35

74.4%

10

102.4%

4

33.8%

19

54.7%

34

62.0%

75

473.8%

26-30

2,628

99.5%

943

136.0%

1,020

102.9%

121

96.6%

64

64.7%

110

115.0%

48

104.1%

203

148.4%

26

69.6%

10

80.6%

13

53.6%

35

79.5%

74

107.3%

100

118.8%

31-35

2,993

104.2%

1,077

125.2%

925

100.6%

197

81.4%

244

106.0%

149

67.2%

104

122.8%

248

119.4%

77

119.2%

13

64.6%

29

127.1%

36

215.6%

113

110.0%

157

112.8%

36-40

3,702

93.1%

1,413

92.5%

920

101.1%

441

102.1%

469

101.8%

222

93.3%

156

114.7%

317

121.4%

76

84.9%

34

113.3%

21

153.1%

49

125.2%

149

109.0%

237

114.8%

41-45

6,116

97.0%

2,617

91.2%

776

113.2%

952

93.6%

862

94.9%

361

110.4%

300

115.1%

534

111.1%

133

107.3%

82

132.0%

64

123.0%

88

106.2%

216

67.4%

357

116.9%

46-50

11,979

101.8%

5,253

94.1%

507

120.3%

1,633

91.9%

1,656

107.2%

447

91.6%

428

105.7%

821

107.4%

178

120.6%

85

88.8%

83

101.2%

131

128.7%

439

96.2%

552

98.4%

51-55

22,124

99.1%

9,749

96.9%

541

147.2%

2,718

95.6%

2,977

102.9%

558

103.0%

662

100.3%

1,357

105.1%

207

97.6%

150

102.6%

156

106.2%

220

114.9%

762

95.8%

988

98.3%

56-60

34,103

99.4%

16,152

99.0%

1,024

134.2%

3,725

100.5%

4,352

99.5%

734

105.1%

906

97.1%

2,149

100.5%

278

111.1%

220

87.5%

198

84.9%

292

83.8%

1,131

92.8%

1,501

109.0%

61-65

48,484

100.0%

23,995

98.7%

1,522

120.6%

4,508

98.2%

6,441

101.9%

844

104.8%

1,276

100.3%

3,319

102.3%

288

84.9%

298

84.1%

265

82.6%

482

115.3%

1,261

93.5%

2,122

94.8%

66-70

62,985

100.4%

32,526

101.0%

2,065

116.7%

4,695

97.0%

8,215

101.6%

759

92.7%

1,653

97.1%

4,485

101.8%

297

84.8%

375

87.4%

282

101.9%

562

110.0%

1,317

98.0%

2,425

96.9%

71-75

79,010

100.3%

37,347

100.9%

2,472

113.6%

5,098

90.2%

10,044

100.9%

813

104.7%

1,781

98.0%

4,852

101.4%

291

113.1%

382

112.2%

271

102.5%

632

110.3%

1,005

111.0%

2,306

93.1%

76-80

103,004

100.1%

37,578

101.2%

2,962

110.7%

5,702

102.5%

12,535

99.2%

657

96.3%

1,826

98.1%

4,936

99.2%

212

90.7%

397

88.6%

262

114.5%

563

98.9%

769

113.4%

1,821

98.6%

81-85

131,733

100.1%

35,105

101.3%

4,020

101.3%

6,564

98.5%

15,019

99.3%

658

102.4%

2,108

99.1%

5,100

99.5%

160

107.0%

351

96.3%

211

112.0%

469

101.6%

448

103.6%

1,262

109.5%

86-90

125,799

100.2%

24,798

101.7%

5,230

97.1%

5,658

102.9%

13,382

100.4%

498

96.7%

1,764

99.2%

4,554

99.6%

106

68.3%

241

109.5%

124

78.6%

367

93.0%

228

85.5%

558

97.1%

91-95

61,162

100.4%

9,766

95.3%

3,742

94.2%

2,563

102.6%

6,842

99.2%

149

89.9%

792

101.5%

2,217

99.1%

30

114.0%

106

103.5%

46

131.0%

173

84.4%

59

118.3%

193

114.9%

96-100

10,451

100.8%

1,639

103.3%

933

75.8%

436

122.2%

1,398

97.2%

11

66.0%

133

111.0%

401

94.7%

3

103.1%

12

153.1%

4

23.4%

51

99.5%

6

168.1%

34

44.0%

101-105

331

108.4%

130

102.4%

68

13.4%

13

136.3%

51

109.4%

5

197.8%

7

108.8%

43

28.6%

0

0.0%

2

110.3%

7

27.6%

1

595.5%

6

52.9%

106-110

8

179.5%

3

143.7%

8

5.1%

1

108.9%

32

31.9%

2

5.2%

1

1.8%

111-115

0

0.0%

5

2.0%

0

0.0%

20

34.6%

2

4.8%

4

42.8%

116-120

4

2.5%

7

25.9%

0

0.0%

2

63.3%

UW

Dur_Grp: 1-1

Dur_Grp: 2-2

Dur_Grp: 3-3

Dur_Grp: 4-5

Dur_Grp: 6-10

Dur_Grp: 11-15

Dur_Grp: 16-20

Dur_Grp: 21-30

Dur_Grp: 31-120

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

NS/1/2

85

78.6%

146

177.8%

214

171.4%

488

62.0%

3,129

100.1%

8,689

98.4%

13,534

98.7%

17,588

100.5%

1,224

118.2%

NS/2/2

133

125.6%

140

96.5%

244

137.9%

849

89.8%

6,018

99.4%

17,296

100.3%

22,596

100.4%

33,941

100.5%

3,371

94.9%

NS/1/3

215

79.0%

332

91.1%

392

97.3%

927

104.2%

2,568

101.0%

1,920

101.4%

642

92.0%

40

51.5%

2

28.8%

NS/2/3

223

99.1%

364

101.1%

451

105.0%

1,209

96.7%

5,262

100.3%

4,819

99.5%

1,467

103.6%

179

63.6%

0

0.0%

NS/3/3

1,152

114.3%

1,611

99.3%

1,970

96.1%

4,485

101.0%

14,771

100.2%

9,774

100.3%

1,252

103.8%

572

78.4%

122

40.7%

NS/1/4

104

104.3%

172

75.9%

243

109.0%

472

99.6%

898

97.6%

465

105.7%

43

65.9%

0

0.0%

0

0.0%

NS/2/4

99

76.3%

142

78.3%

181

80.6%

482

107.3%

1,095

97.9%

629

108.8%

136

97.4%

2

74.9%

0

0.0%

NS/3/4

65

71.8%

127

98.7%

144

91.0%

334

112.1%

778

97.1%

554

110.1%

33

101.6%

0

0.0%

NS/4/4

231

137.9%

306

112.8%

383

99.5%

848

105.5%

1,604

99.6%

639

103.5%

85

65.4%

66

19.3%

18

21.3%

NS/U/U

1,909

126.7%

2,351

101.9%

2,761

110.0%

6,404

103.9%

19,610

99.2%

38,118

99.5%

92,885

99.7%

444,936

100.0%

101,189

100.2%

S/1/2

122

142.1%

176

76.2%

215

72.1%

487

88.7%

1,682

95.1%

1,916

94.7%

1,651

106.8%

1,643

108.4%

120

167.6%

S/2/2

437

112.6%

474

130.8%

540

78.4%

1,079

86.6%

2,846

95.7%

3,016

103.8%

2,847

106.0%

3,115

107.4%

347

86.5%

S/U/U

466

225.9%

525

81.3%

643

90.1%

1,545

104.3%

5,164

104.6%

12,728

97.9%

31,327

99.0%

151,687

100.3%

37,600

100.4%

U/U/U

107

94.9%

79

52.1%

95

70.0%

198

75.7%

796

129.4%

1,198

100.8%

2,109

106.4%

16,791

101.8%

9,404

96.0%

AA_Grp

Dur_Grp: 1-1

Dur_Grp: 2-2

Dur_Grp: 3-3

Dur_Grp: 4-5

Dur_Grp: 6-10

Dur_Grp: 11-15

Dur_Grp: 16-20

Dur_Grp: 21-30

Dur_Grp: 31-120

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

Deaths

Ratio

0-17

173

68.1%

105

49.6%

104

92.0%

140

73.3%

358

72.1%

390

86.1%

318

68.2%

18-25

270

84.7%

222

142.7%

194

85.5%

388

90.6%

858

124.1%

751

92.3%

1,338

89.3%

2,062

98.0%

26-30

144

111.2%

154

82.3%

173

139.5%

310

134.0%

668

99.8%

674

104.2%

617

97.4%

2,513

102.2%

142

102.6%

31-35

167

127.1%

179

94.3%

170

93.5%

367

122.4%

905

99.4%

990

111.8%

841

96.7%

2,006

95.6%

737

106.6%

36-40

179

188.7%

197

79.6%

182

119.0%

407

89.9%

1,045

105.0%

1,401

95.8%

1,515

98.3%

2,712

97.1%

568

92.2%

41-45

241

89.4%

295

123.8%

324

122.4%

603

103.0%

1,537

100.7%

2,110

103.1%

2,708

95.5%

4,940

93.2%

700

96.2%

46-50

368

110.1%

368

79.0%

425

111.5%

810

91.9%

2,164

103.1%

3,160

102.9%

4,617

102.3%

11,001

97.6%

1,279

103.6%

51-55

475

96.9%

538

80.7%

659

103.1%

1,253

103.1%

3,316

97.7%

4,811

101.8%

7,390

100.4%

21,380

99.7%

3,347

101.8%

56-60

655

135.9%

812

107.7%

927

91.1%

1,786

94.9%

4,650

101.7%

6,376

99.4%

10,273

98.6%

34,227

99.4%

7,059

98.6%

61-65

856

101.0%

1,020

109.6%

1,238

92.0%

2,522

96.5%

6,282

99.1%

8,325

98.7%

13,454

99.6%

49,923

100.4%

11,485

102.3%

66-70

735

108.8%

1,135

86.4%

1,419

99.5%

3,162

98.6%

7,861

98.9%

10,137

98.9%

16,691

101.0%

65,377

100.4%

16,124

101.3%

71-75

483

80.5%

842

106.8%

1,086

91.1%

2,945

103.1%

9,258

100.9%

12,280

99.1%

19,613

99.1%

80,264

100.3%

19,533

101.4%

76-80

295

95.9%

549

98.8%

747

100.3%

2,208

102.4%

9,054

99.4%

14,574

100.7%

25,716

99.2%

97,363

100.1%

22,718

101.2%

81-85

146

98.8%

321

102.7%

532

114.9%

1,761

97.0%

8,778

99.7%

15,015

100.1%

29,833

99.4%

120,517

100.2%

26,305

101.1%

86-90

59

217.0%

111

153.6%

211

98.9%

897

105.2%

6,832

99.8%

12,830

99.7%

23,281

100.5%

113,684

100.2%

25,402

99.1%

91-95

50

127.3%

44

278.0%

50

95.8%

188

68.8%

2,406

99.1%

6,698

99.6%

10,510

101.0%

53,214

100.5%

14,680

101.0%

96-100

52

143.3%

53

96.0%

30

164.7%

48

172.0%

243

108.1%

1,196

102.1%

1,805

99.9%

8,967

98.8%

3,118

90.4%

101-105

5

129.1%

12

334.1%

6

128.2%

43

90.6%

82

149.0%

356

68.6%

160

34.7%

106-110

5

70.0%

32

42.9%

18

9.2%

111-115

0

0.0%

0

0.0%

20

31.4%

11

3.2%

116-120

2

24.8%

11

3.7%

As with other models, tor the bivariate fit checks, the fit can be great in some cases and poor in others. This could be due to a need for higher level interactions.

9 Impact on Trend of Adjusting for Other Variables

Even though the goal of this work is to support other analysis, it is worth examining the impact on trend of controlling for other variables. In the table below, it can be seen that there is a narrower spread of ratios between 2011 and 2017, 103.5% to 98.1% for the ratios against the model, and 98.3% to 89.4% for the ratios against the 2015 VBT. This is a meaningful reduction in residual trend.

Code
ilec_arrow_src <- '/workspace/Projects/ILEC/VBT/Data/ilecdata'

dat.arrow <- arrow::open_dataset(ilec_arrow_src)

AA.brks <- c(-1,17,
             seq(25,115,5))
AA.lbls <- paste0(
  AA.brks[1:(length(AA.brks)-1)]+1,
  "-",
  AA.brks[2:length(AA.brks)]
)

Dur.brks <- c(0,1,2,3,5,10,15,20,30,100)
Dur.lbls <- paste0(
  Dur.brks[1:(length(Dur.brks)-1)]+1,
  "-",
  Dur.brks[2:length(Dur.brks)]
)

dat.arrow %>%
  filter(Observation_Year >= 2011 
         & Observation_Year <= 2017 
         & Insurance_Plan == 'Term'
         & SOA_Post_Lvl_Ind != 'PLT') %>%
  #select(!Observation_Year) %>%
  group_by(
    Sex,
    Smoker_Status,
    Attained_Age,
    Duration,
    Face_Amount_Band,
    SOA_Post_Lvl_Ind,
    Number_of_Pfd_Classes,
    Preferred_Class,
    Observation_Year
  ) %>%
  summarize(
    Death_Count=sum(Death_Count),
    Policies_Exposed=sum(Policies_Exposed),
    ExpDth_Cnt_VBT2015=sum(ExpDth_Cnt_VBT2015),
    Death_Claim_Amount=sum(Death_Claim_Amount),
    Amount_Exposed=sum(Amount_Exposed),
    ExpDth_Amt_VBT2015=sum(ExpDth_Amt_VBT2015)
    ) %>%
  collect() %>%
  as.data.table() %>%
  mutate(
    Smoker_Status=as.character(Smoker_Status),
    Preferred_Class=as.character(Preferred_Class),
    Number_of_Pfd_Classes=as.character(Number_of_Pfd_Classes)
  ) %>%
  mutate(Smoker_Status=ifelse(is.na(Smoker_Status),"U",Smoker_Status),
         Number_of_Pfd_Classes=ifelse(is.na(Number_of_Pfd_Classes),"U",
                                      Number_of_Pfd_Classes),
         Preferred_Class=ifelse(is.na(Preferred_Class),"U",Preferred_Class),
         UW=paste0(Smoker_Status,"/",Preferred_Class,"/",Number_of_Pfd_Classes),
         AA_Grp=cut(Attained_Age,breaks=AA.brks,labels = AA.lbls),
         Dur_Grp=cut(Duration,breaks=Dur.brks,labels=Dur.lbls),
         Subset="TermNoPLT",
         Face_Amount_Band=fct_relabel(
          Face_Amount_Band,
          #function(.) sub(":"," -",.,fixed=T)
          function(.) fa.remap[Face_Amount_Band.Old==.,Face_Amount_Band.New]
          )
         ) %>%
  left_join(
    y=rbind(dat.term.noplt.lt100k.grid,dat.term.noplt.100kp.grid)
  ) %>%
  group_by(
    Subset,
    Observation_Year
  ) %>%
  summarize(
    Death_Count=sum(Death_Count),
    Death_Amount=sum(Death_Claim_Amount),
    TabularClaimsAmount=sum(ExpDth_Amt_VBT2015),
    ModelClaimsAmount=sum(ExpDth_Amt_VBT2015*Factor)
  ) %>%
  data.table() ->
  dat.term.noplt.2




dat.arrow %>%
  filter(Observation_Year >= 2011 
         & Observation_Year <= 2017 
         & Insurance_Plan == 'Term'
         & SOA_Post_Lvl_Ind == 'PLT') %>%
  #select(!Observation_Year) %>%
  group_by(
    Sex,
    Smoker_Status,
    Attained_Age,
    Duration,
    Face_Amount_Band,
    SOA_Antp_Lvl_TP,
    Number_of_Pfd_Classes,
    Preferred_Class,
    Observation_Year
  ) %>%
  summarize(
    Death_Count=sum(Death_Count),
    Policies_Exposed=sum(Policies_Exposed),
    ExpDth_Cnt_VBT2015=sum(ExpDth_Cnt_VBT2015),
    Death_Claim_Amount=sum(Death_Claim_Amount),
    Amount_Exposed=sum(Amount_Exposed),
    ExpDth_Amt_VBT2015=sum(ExpDth_Amt_VBT2015)
    ) %>%
  collect() %>%
  as.data.table() %>%
  mutate(
    Smoker_Status=as.character(Smoker_Status),
    Preferred_Class=as.character(Preferred_Class),
    Number_of_Pfd_Classes=as.character(Number_of_Pfd_Classes)
  ) %>%
  mutate(Smoker_Status=ifelse(is.na(Smoker_Status),"U",Smoker_Status),
         Number_of_Pfd_Classes=ifelse(is.na(Number_of_Pfd_Classes),"U",
                                      Number_of_Pfd_Classes),
         Preferred_Class=ifelse(is.na(Preferred_Class),"U",Preferred_Class),
         UW=paste0(Smoker_Status,"/",Preferred_Class,"/",Number_of_Pfd_Classes),
         AA_Grp=cut(Attained_Age,breaks=AA.brks,labels = AA.lbls),
         Dur_Grp=cut(Duration,breaks=Dur.brks,labels=Dur.lbls),
         Subset="TermPLT",
         Face_Amount_Band=fct_relabel(
          Face_Amount_Band,
          #function(.) sub(":"," -",.,fixed=T)
          function(.) fa.remap[Face_Amount_Band.Old==.,Face_Amount_Band.New]
          )
         )%>%
  left_join(
    y=dat.term.plt.grid
  )  %>%
  group_by(
    Subset,
    Observation_Year
  ) %>%
  summarize(
    Death_Count=sum(Death_Count),
    Death_Amount=sum(Death_Claim_Amount),
    TabularClaimsAmount=sum(ExpDth_Amt_VBT2015),
    ModelClaimsAmount=sum(ExpDth_Amt_VBT2015*Factor)
  ) ->
  dat.term.plt.2

AA.brks <- c(-1,17,
             seq(25,120,5))
AA.lbls <- paste0(
  AA.brks[1:(length(AA.brks)-1)]+1,
  "-",
  AA.brks[2:length(AA.brks)]
)

Dur.brks <- c(0,1,2,3,5,10,15,20,30,120)
Dur.lbls <- paste0(
  Dur.brks[1:(length(Dur.brks)-1)]+1,
  "-",
  Dur.brks[2:length(Dur.brks)]
)

dat.arrow %>%
  filter(Observation_Year >= 2011 
         & Observation_Year <= 2017 
         & Insurance_Plan != 'Term') %>%
  #select(!Observation_Year) %>%
  group_by(
    Sex,
    Smoker_Status,
    Attained_Age,
    Duration,
    Face_Amount_Band,
    SOA_Post_Lvl_Ind,
    Number_of_Pfd_Classes,
    Preferred_Class,
    Observation_Year,
    Insurance_Plan
  ) %>%
  summarize(
    Death_Count=sum(Death_Count),
    Policies_Exposed=sum(Policies_Exposed),
    ExpDth_Cnt_VBT2015=sum(ExpDth_Cnt_VBT2015),
    Death_Claim_Amount=sum(Death_Claim_Amount),
    Amount_Exposed=sum(Amount_Exposed),
    ExpDth_Amt_VBT2015=sum(ExpDth_Amt_VBT2015)
    ) %>%
  collect() %>%
  as.data.table() %>%
  mutate(
    Smoker_Status=as.character(Smoker_Status),
    Preferred_Class=as.character(Preferred_Class),
    Number_of_Pfd_Classes=as.character(Number_of_Pfd_Classes)
  ) %>%
  mutate(Smoker_Status=ifelse(is.na(Smoker_Status),"U",Smoker_Status),
         Number_of_Pfd_Classes=ifelse(is.na(Number_of_Pfd_Classes),"U",
                                      Number_of_Pfd_Classes),
         Preferred_Class=ifelse(is.na(Preferred_Class),"U",Preferred_Class),
         UW=paste0(Smoker_Status,"/",Preferred_Class,"/",Number_of_Pfd_Classes),
         AA_Grp=cut(Attained_Age,breaks=AA.brks,labels = AA.lbls),
         Dur_Grp=cut(Duration,breaks=Dur.brks,labels=Dur.lbls),
         Subset=ifelse(Insurance_Plan=="Perm" & UW=="U/U/U", "Perm_Unk","Perm_Knw"),
         Face_Amount_Band=fct_relabel(
          Face_Amount_Band,
          #function(.) sub(":"," -",.,fixed=T)
          function(.) fa.remap[Face_Amount_Band.Old==.,Face_Amount_Band.New]
          )
         ) %>%
  left_join(
    y=dat.perm.knw.grid[!(Insurance_Plan=="Perm" & UW == "U/U/U")]
  ) %>%
  rename(
    Factor_KNW = Factor
  ) %>%
  left_join(
    y=dat.perm.unk.grid
  ) %>%
  mutate(
    Factor = ifelse(is.na(Factor_KNW),Factor,Factor_KNW)
  ) %>%
  group_by(
    Subset,
    Observation_Year
  ) %>%
  summarize(
    Death_Count=sum(Death_Count),
    Death_Amount=sum(Death_Claim_Amount),
    TabularClaimsAmount=sum(ExpDth_Amt_VBT2015),
    ModelClaimsAmount=sum(ExpDth_Amt_VBT2015*Factor)
  ) ->
  dat.perm.2

dat.illustration <- rbind(
  dat.term.noplt.2,
  dat.term.plt.2,
  dat.perm.2
)

rm(dat.term.noplt.2, dat.term.plt.2, dat.perm.2)

dat.illustration %>%
  group_by(Observation_Year) %>%
  summarize(
    Death_Count=sum(Death_Count),
    Death_Amount=sum(Death_Amount),
    TabularClaimsAmount=sum(TabularClaimsAmount),
    ModelClaimsAmount=sum(ModelClaimsAmount),
    A_E_Tbl=Death_Amount/TabularClaimsAmount,
    A_E_Mod=Death_Amount/ModelClaimsAmount
  ) %>%
  flextable() %>%
  set_formatter(
    A_E_Tbl = function(x) {
      if(is.numeric(x))
        sprintf( "%.1f%%", x*100 )
      else
        x
      },
    A_E_Mod = function(x) {
      if(is.numeric(x))
        sprintf( "%.1f%%", x*100 )
      else
        x
      }
  ) %>%
  colformat_num(j=c("Death_Count","Death_Amount",
                    "TabularClaimsAmount","ModelClaimsAmount") ) %>%
  colformat_int(j="Observation_Year",big.mark="") %>%
  set_header_labels(
    Observation_Year = "Observation Year",
    Death_Count = "Death Count",
    Death_Amount = "Death Amount",
    TabularClaimsAmount="15VBT Tabular Claims",
    ModelClaimsAmount = "Model Predicted Claims",
    A_E_Tbl = "Actual-to-15VBT",
    A_E_Mod = "Actual-to-Model"
  )

Observation Year

Death Count

Death Amount

15VBT Tabular Claims

Model Predicted Claims

Actual-to-15VBT

Actual-to-Model

2011

563,694

26,101,825,065

26,560,415,265

25,224,268,217

98.3%

103.5%

2012

537,286

27,828,587,056

28,843,616,822

27,075,644,852

96.5%

102.8%

2013

554,199

30,043,247,445

32,019,431,009

30,071,134,913

93.8%

99.9%

2014

560,393

32,777,977,745

35,396,309,259

33,012,977,593

92.6%

99.3%

2015

565,853

35,479,567,171

38,423,917,677

35,522,249,984

92.3%

99.9%

2016

552,127

37,188,498,560

41,266,090,988

37,745,327,915

90.1%

98.5%

2017

558,579

38,975,688,945

43,595,587,331

39,743,791,962

89.4%

98.1%

The ratios with fitted trend lines from a GLM bear this out visually. The fitted GLMs show an average 1.5% decline per annum in ratios against the 2015 VBT versus an average 0.8% decline per annum of the ratios against the model.

Code
mod.glm.tbl <- glm(
  Death_Amount ~ Observation_Year,
  data=dat.illustration,
  family=quasipoisson,
  offset=log(TabularClaimsAmount)
)

mod.glm.mdl <- glm(
  Death_Amount ~ Observation_Year,
  data=dat.illustration,
  family=quasipoisson,
  offset=log(ModelClaimsAmount)
)

dat.illustration %>%
  group_by(Observation_Year) %>%
  summarize(
    Death_Count=sum(Death_Count),
    Death_Amount=sum(Death_Amount),
    TabularClaimsAmount=sum(TabularClaimsAmount),
    ModelClaimsAmount=sum(ModelClaimsAmount),
    A_E_Tbl=Death_Amount/TabularClaimsAmount,
    A_E_Mod=Death_Amount/ModelClaimsAmount
  ) %>%
  ggplot(aes(x=Observation_Year)) +
  geom_line(aes(y=A_E_Tbl)) +
  geom_line(aes(y=A_E_Mod),color="green") +
  geom_smooth(aes(y=A_E_Tbl,weight=TabularClaimsAmount),
              method.args=list(family="quasipoisson"),
              method="glm",linetype=2) +
  geom_smooth(aes(y=A_E_Mod,weight=ModelClaimsAmount),
              method.args=list(family="quasipoisson"),
              method="glm",color="green",linetype=2) +
  annotate(geom="text",
           label=paste0("Residual Trend vs. 15VBT: ", sprintf( "%.1f%%", (exp(mod.glm.tbl$coefficients[2])-1)*100 ), " p.a."),
           x=2013,y=.92) +
  annotate(geom="text",
           label=paste0("Residual Trend vs. Model: ", sprintf( "%.1f%%", (exp(mod.glm.mdl$coefficients[2])-1)*100 ), " p.a."),
           x=2015,y=1.02) + 
  scale_y_continuous(name="Actual vs. Model or Table",labels = scales::percent) +
  scale_x_continuous(name="Observation Year") +
  ggtitle(label="Actual Experience versus Models",subtitle="with GLM trends") +
  theme_minimal()

In the chart below, we can see a substantial year-over-year growth in face amounts exposed for Term, Perm, and UL/VL. The reader is invited to explore the data themselves to understand prevalence shifts, which is beyond the scope of this work.

Code
dat.arrow %>%
  filter(Observation_Year >= 2011 
         & Observation_Year <= 2017) %>%
  group_by(
    Insurance_Plan,
    Face_Amount_Band,
    Observation_Year
  ) %>%
  summarise(
    Amount_Exposed=sum(Amount_Exposed)
  ) %>% 
  collect() %>%
  ungroup(Face_Amount_Band) %>%
  mutate(Amount_Exposed=Amount_Exposed/sum(Amount_Exposed),
         Face_Amount_Band=fct_relabel(
           Face_Amount_Band,
           function(.) substr(.,1,2)
         )
    ) %>%
  data.table() %>%
  ggplot(aes(x=Observation_Year)) +
  geom_bar(aes(y=Amount_Exposed),stat="identity") + 
  facet_grid(rows=vars(Insurance_Plan),cols=vars(Face_Amount_Band)) +
  theme_minimal() +
  ggtitle("Exposure Trends 2011-17", 
          subtitle = "by Face Amount Band and Insurance Plan") +
  scale_y_continuous(name="Amount Exposed (% of Total Exposure)",
                     labels = scales::percent) +
  scale_x_continuous(name="Observation Year") +
  theme(axis.text.x = element_text(angle=90))

10 Running the Workbook

To run this workbook, you will need the supporting R functions and a compatible dataset. The supporting R functions are embedded here if for some reason they are not provided with this document. The dataset is an Arrow dataset based on the standard download from the Society of Actuaries. The CSV was converted to Parquet files split by gender and observation year.

Code
# catboost.shapviz.R
shapviz.catboost.Model <- function(object, X_pred, X = X_pred, collapse = NULL, ...) {
  if (!requireNamespace("catboost", quietly = TRUE)) {
    stop("Package 'catboost' not installed")
  }
  stopifnot(
    "X must be a matrix or data.frame. It can't be an object of class catboost.Pool" =
      is.matrix(X) || is.data.frame(X),
    "X_pred must be a matrix, a data.frame, or a catboost.Pool" =
      is.matrix(X_pred) || is.data.frame(X_pred) || inherits(X_pred, "catboost.Pool"),
    "X_pred must have column names" = !is.null(colnames(X_pred))
  )
  
  if (!inherits(X_pred, "catboost.Pool")) {
    X_pred <- catboost.load_pool(X_pred)
  }
  
  S <- catboost.get_feature_importance(object, X_pred, type = "ShapValues", ...)
  
  # Call matrix method
  pp <- ncol(X_pred) + 1
  baseline <- S[1, pp]
  S <- S[, -pp, drop = FALSE]
  colnames(S) <- colnames(X_pred)
  shapviz(S, X = X, baseline = baseline, collapse = collapse)
}

#glmnet_support.R
prepELData <- function( formula, data, predictors, response, offset=NULL, weights=NULL, 
                        foldid=NULL, useSparse=FALSE, dropunused=FALSE ) {
  
  if(is.null(formula))
    stop("formula must be provided")
  if(is.null(data))
    stop("data must be provided")
  if(is.null(response))
    stop("response must be provided")
  
  # if(useSparse) {
  #   outMat <- sparse.model.matrix(
  #     formula,
  #     data[,..predictors]
  #   )
  # } else {
  #   outMat <- model.matrix(
  #     formula,
  #     data[,..predictors]
  #   )
  # }
  
  outMat <- model.Matrix(
    formula,
    data[,..predictors],
    sparse=useSparse,
    drop.unused.levels = dropunused
  )

  outResp <- as.matrix(data[[response]])
  
  if(is.null(offset))
    outOffset <- NULL
  else
    outOffset <- as.matrix(data[[offset]])
  
  if(is.null(weights))
    outWeights <- NULL
  else
    outWeights <- as.matrix(data[[weights]])
  
  if(is.null(foldid))
    outFoldID <- NULL
  else
    outFoldID <- as.matrix(data[[foldid]])
  
  list(input.matrix=outMat,
       response=outResp,
       offset = outOffset,
       weights = outWeights,
       foldid = outFoldID)
}

fitCVGLMNet <- function(datalist,alpha=0.5,
                        nfolds=ifelse(is.null(datalist$foldid),5,max(datalist$foldid))) {
  cl <- makeCluster(nfolds)
  registerDoParallel(cl)
  
  #glmnet.control(itrace = 1)
  
  cvfit <- try(
    {
      cv.glmnet(
        x=datalist$input.matrix,
        y=datalist$response,
        offset=datalist$offset,
        weights=datalist$weights,
        family="poisson",
        alpha=alpha,
        nfolds=nfolds,
        foldid = datalist$foldid,
        parallel=T,
        standardize=F)
    }
  )

  
  stopCluster(cl)
  
  cvfit
}

# https://stackoverflow.com/questions/6457290/how-to-check-the-amount-of-ram
available_memory <- function()
{
  
  # Get operating system
  OS <- tolower(Sys.info()["sysname"])
  
  # Branch based on OS
  if(OS == "windows"){ # Windows
    
    # System information
    system_info <- system("systeminfo", intern = TRUE)
    
    # Get available memory
    value <- system_info[
      grep("Available Physical Memory", system_info)
    ]
    
    # Remove extraneous information
    value <- gsub("Available Physical Memory: ", "", value)
    value <- gsub("\\,", "", value)
    
    # Convert to bytes
    value_split <- unlist(strsplit(value, split = " "))
    
    # Check for second value
    bytes <- as.numeric(value_split[1]) * switch(
      value_split[2],
      "KB" = 1e03,
      "MB" = 1e06,
      "GB" = 1e09
    )
    
  }else if(OS == "linux"){ # Linux
    
    # Split system information
    info_split <- strsplit(system("free -b", intern = TRUE), split = " ")
    
    # Remove "Mem:" and "Swap:"
    info_split <- lapply(info_split, function(x){gsub("Mem:", "", x)})
    info_split <- lapply(info_split, function(x){gsub("Swap:", "", x)})
    
    # Get actual values
    info_split <- lapply(info_split, function(x){x[x != ""]})
    
    # Bind values
    info_split <- do.call(rbind, info_split[1:2])
    
    # Get free values
    bytes <- as.numeric(info_split[2, info_split[1,] == "free"])
    
  }else{ # Mac
    
    # System information
    system_info <- system("top -l 1 -s 0 | grep PhysMem", intern = TRUE)
    
    # Get everything after comma
    unused <- gsub(" .*,", "", system_info)
    
    # Get values only
    value <- gsub("PhysMem: ", "", unused)
    value <- gsub(" unused.", "", value)
    
    # Check for bytes
    if(grepl("M", value)){
      bytes <- as.numeric(gsub("M", "", value)) * 1e06
    }else if(grepl("G", value)){
      bytes <- as.numeric(gsub("G", "", value)) * 1e09
    }else if(grepl("K", value)){
      bytes <- as.numeric(gsub("K", "", value)) * 1e03
    }
    
  }
  
  # Return bytes
  return(bytes)
  
}

reformatCoefs <- function(cvfit,pred.cols,s="lambda.min") {
  coef(cvfit,s=s) %>%
    as.matrix() %>%
    as.data.table(keep.rownames=T) %>%
    setnames(c("rn","s1"),c("CoefName","Coef")) ->
    coefs 
  
  coefs[,c("Feature1","Feature2") := tstrsplit(CoefName,":")]

  walk(pred.cols,\(x) 
       coefs[startsWith(Feature1,x),
                        `:=`(Feature1Name=x
                        )]
  )
  
  coefs[,
                   Feature1Level:=mapply(
                     function(f,fname) {
                       str_sub(
                         f,
                         end=-1,
                         start=nchar(fname)-nchar(f)
                       )
                     },
                     Feature1,
                     Feature1Name,
                     SIMPLIFY = "vector"
                   )]
  
  walk(pred.cols,\(x) 
       coefs[startsWith(Feature2,x),
                        `:=`(Feature2Name=x
                        )]
  )
  
  coefs[,
                   Feature2Level:=mapply(
                     function(f,fname) {
                       str_sub(
                         f,
                         end=-1,
                         start=nchar(fname)-nchar(f)
                       )
                     },
                     Feature2,
                     Feature2Name,
                     SIMPLIFY = "vector"
                   )]
  
  coefs[Feature1=="(Intercept)",
                   `:=`(Feature1Name="(Intercept)",
                        Feature1Level="(Intercept)")]
  
  coefs[,`:=`(Feature1=NULL, Feature2=NULL)]
  
  coefs
}

plotCVNetCoefs <- function(
    model.grid,
    vars,
    factorcol,
    pred.cols,
    whichlevel = "mid"
) {
  
  v1 <- sym(vars[1])
  v2 <- sym(vars[2])
  vf <- sym(factorcol)
  
  model.grid[,..pred.cols] %>%
    lapply(levels) %>%
    imap(.f=\(x,idx) {
      if(idx %in% vars[1:2]) {
        retval <- x
      } else {
        if(whichlevel == "first") 
          retval <- x[1]
        else if(whichlevel == "last")
          retval <- x[length(x)]
        else if(whichlevel == "mid")
          retval <- x[length(x)/2]
      }
      retval
    }) %>%
    expand.grid() %>%
    setDT() ->
    filter.frame
  
  filter.frame %>%
    select(!c(!!v1,!!v2)) %>%
    distinct() %>%
    imap( .f = \(x,idx) paste0(idx,": ",x)) %>%
    paste(collapse=", ") %>%
    paste0("Other fixed variables - ",.) ->
    st
  
  model.grid %>%
    inner_join(filter.frame) %>%
    ggplot(aes(x=!!v1,y=!!vf)) +
    geom_point() +
    facet_wrap(vars(!!v2)) +
    scale_y_continuous(labels = scales::percent) +
    geom_hline(yintercept=1,linetype=2) +
    ggtitle(label=paste0("Coefficients for ",vars[1], " by ", vars[2]),
            subtitle = st) +
    theme_minimal() + 
    theme(
      axis.text.x = element_text(angle=90)
    )
}


tableCVNetCoefs <- function(
    model.grid,
    vars,
    factorcol,
    pred.cols,
    whichlevel = "mid"
) {
  
  v1 <- sym(vars[1])
  v2 <- sym(vars[2])
  vf <- sym(factorcol)
  
  model.grid[,..pred.cols] %>%
    lapply(levels) %>%
    imap(.f=\(x,idx) {
      if(idx %in% vars[1:2]) {
        retval <- x
      } else {
        if(whichlevel == "first") 
          retval <- x[1]
        else if(whichlevel == "last")
          retval <- x[length(x)]
        else if(whichlevel == "mid")
          retval <- x[length(x)/2]
      }
      retval
    }) %>%
    expand.grid() %>%
    setDT() ->
    filter.frame
  
  filter.frame %>%
    select(!c(!!v1,!!v2)) %>%
    distinct() %>%
    imap( .f = \(x,idx) paste0(idx,": ",x)) %>%
    paste(collapse=", ") %>%
    paste0("Other fixed variables - ",.) ->
    st
  
  model.grid %>%
    inner_join(filter.frame) %>%
    select(!!v1,!!v2,Factor) %>%
    pivot_wider(names_from=!!v2,
                values_from=Factor)
}

# support_fns.R
generate_tabset <- function(tabs,
                            tabtitle,
                            tablevel = 1) {
  # markdown <- paste0(
  #   paste(rep("#",tablevel),collapse=""),
  #   " ",
  #   tabtitle,
  #   " ",
  #   "{.tabset}\n\n")
  
  # markdown <- "::: panel-tabset"
  markdown <- ""
  
  for (i in seq_along(tabs)) {
    title <- names(tabs)[i]
    content <- tabs[[i]]
    markdown <- paste0(markdown, 
                       paste(rep("#",tablevel + 1),collapse=""),
                       " ", 
                       title, 
                       "\n\n", 
                       content, 
                       "\n\n")
  }
  # markdown <- paste0(markdown,"\n",":::")
  return(markdown)
}

The dataset is extracted from the ILEC data. Provided variable names and their content are identical and that the data objects are stored in RDS files of the appropriate names, then the workbook should run as is.

Computation is not onerous. A GPU was used for gradient boosting in anticipation of needing substantial computing power. Early attempts greatly benefited from it. However, as modeling progressed, the gradient boosting models became simple enough that it could run reasonably well on CPU. For elastic net, the use of sparse matrices ensures that memory should not be an issue except in the most basic environment. CPU usage is a different matter. By default, we use 10 cores for cross-validation. If 10 cores are available, the elastic net models typically require only a few minutes to run. If you do not have 10 cores, adjust your expectations accordingly.

The workbook, when executed, will carry out the following:

  1. Load the data and apply transformations.

  2. If not already made, fit Catboost models, and exhibit results.

  3. If not already made, fit elastic net models, and exhibit results.

  4. Generate factor tables for attaching to experience data.

11 About the Society of Actuaries Research Institute

Serving as the research arm of the Society of Actuaries (SOA), the SOA Research Institute provides objective, data-driven research bringing together tried and true practices and future-focused approaches to address societal challenges and your business needs. The Institute provides trusted knowledge, extensive experience and new technologies to help effectively identify, predict and manage risks.

Representing the thousands of actuaries who help conduct critical research, the SOA Research Institute provides clarity and solutions on risks and societal challenges. The Institute connects actuaries, academics, employers, the insurance industry, regulators, research partners, foundations and research institutions, sponsors and non-governmental organizations, building an effective network which provides support, knowledge and expertise regarding the management of risk to benefit the industry and the public.

Managed by experienced actuaries and research experts from a broad range of industries, the SOA Research Institute creates, funds, develops and distributes research to elevate actuaries as leaders in measuring and managing risk. These efforts include studies, essay collections, webcasts, research papers, survey reports, and original research on topics impacting society.

Harnessing its peer-reviewed research, leading-edge technologies, new data tools and innovative practices, the Institute seeks to understand the underlying causes of risk and the possible outcomes. The Institute develops objective research spanning a variety of topics with its strategic research programs: aging and retirement; actuarial innovation and technology; mortality and longevity; diversity, equity and inclusion; health care cost trends; and catastrophe and climate risk. The Institute has a large volume of topical research available, including an expanding collection of international and market-specific research, experience studies, models and timely research.

Society of Actuaries Research Institute
8770 W Bryn Mawr Ave, Suite 1000
Chicago, IL 60631
www.SOA.org

12 Session Information

Information on the environment and package versions used for this analysis.

Code
sessionInfo()
R version 4.3.3 (2024-02-29)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.4 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so;  LAPACK version 3.10.0

locale:
[1] C

time zone: Etc/UTC
tzcode source: system (glibc)

attached base packages:
[1] parallel  stats     graphics  grDevices utils     datasets  methods  
[8] base     

other attached packages:
 [1] arrow_15.0.1       ggridges_0.5.6     MatrixModels_0.5-3 ftExtra_0.6.2     
 [5] flextable_0.9.5    openxlsx_4.2.5.2   glmnet_4.1-8       Matrix_1.6-3      
 [9] doParallel_1.0.17  iterators_1.0.14   foreach_1.5.2      patchwork_1.2.0   
[13] shapviz_0.9.3      dtplyr_1.3.1       lubridate_1.9.3    forcats_1.0.0     
[17] stringr_1.5.1      dplyr_1.1.4        purrr_1.0.2        readr_2.1.5       
[21] tidyr_1.3.1        tibble_3.2.1       ggplot2_3.5.0      tidyverse_2.0.0   
[25] data.table_1.15.2  catboost_1.2.2    

loaded via a namespace (and not attached):
 [1] tidyselect_1.2.1        viridisLite_0.4.2       farver_2.1.1           
 [4] fastmap_1.1.1           fontquiver_0.2.1        promises_1.2.1         
 [7] digest_0.6.35           timechange_0.3.0        mime_0.12              
[10] lifecycle_1.0.4         gfonts_0.2.0            ellipsis_0.3.2         
[13] survival_3.5-7          magrittr_2.0.3          compiler_4.3.3         
[16] rlang_1.1.3             tools_4.3.3             utf8_1.2.4             
[19] yaml_2.3.8              knitr_1.45              labeling_0.4.3         
[22] askpass_1.2.0           htmlwidgets_1.6.4       bit_4.0.5              
[25] xgboost_1.7.7.1         curl_5.2.1              xml2_1.3.6             
[28] httpcode_0.3.0          withr_3.0.0             grid_4.3.3             
[31] fansi_1.0.6             gdtools_0.3.7           xtable_1.8-4           
[34] colorspace_2.1-0        scales_1.3.0            crul_1.4.0             
[37] cli_3.6.2               rmarkdown_2.26          crayon_1.5.2           
[40] ragg_1.3.0              generics_0.1.3          rstudioapi_0.15.0      
[43] tzdb_0.4.0              splines_4.3.3           assertthat_0.2.1       
[46] vctrs_0.6.5             jsonlite_1.8.8          fontBitstreamVera_0.1.1
[49] hms_1.1.3               bit64_4.0.5             systemfonts_1.0.6      
[52] glue_1.7.0              codetools_0.2-19        stringi_1.8.3          
[55] shape_1.4.6.1           gtable_0.3.4            later_1.3.2            
[58] munsell_0.5.0           pillar_1.9.0            htmltools_0.5.7        
[61] openssl_2.1.1           R6_2.5.1                textshaping_0.3.7      
[64] evaluate_0.23           shiny_1.8.0             lattice_0.22-5         
[67] fontLiberation_0.1.0    httpuv_1.6.14           uuid_1.2-0             
[70] Rcpp_1.0.12             zip_2.3.1               nlme_3.1-163           
[73] mgcv_1.9-1              officer_0.6.5           xfun_0.42              
[76] pkgconfig_2.0.3        

13 References

Barrett, Tyson, Matt Dowle, Arun Srinivasan, Jan Gorecki, Michael Chirico, and Toby Hocking. 2024. Data.table: Extension of ‘Data.frame‘. https://CRAN.R-project.org/package=data.table.
Bates, Douglas, and Martin Maechler. 2023. MatrixModels: Modelling with Sparse and Dense Matrices. https://CRAN.R-project.org/package=MatrixModels.
Corporation, Microsoft, and Steve Weston. 2022. doParallel: Foreach Parallel Adaptor for the ’Parallel’ Package. https://CRAN.R-project.org/package=doParallel.
Dorogush, Anna Veronika, Vasily Ershov, and Andrey Gulin. 2018. “CatBoost: Gradient Boosting with Categorical Features Support.” https://arxiv.org/abs/1810.11363.
Gohel, David, and Panagiotis Skintzos. 2024. Flextable: Functions for Tabular Reporting. https://CRAN.R-project.org/package=flextable.
Mayer, Michael. 2024. Shapviz: SHAP Visualizations. https://CRAN.R-project.org/package=shapviz.
Pedersen, Thomas Lin. 2024. Patchwork: The Composer of Plots. https://CRAN.R-project.org/package=patchwork.
Prokhorenkova, Liudmila, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. 2017. “CatBoost: Unbiased Boosting with Categorical Features.” https://arxiv.org/abs/1706.09516.
R Core Team. 2024. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. https://www.R-project.org/.
Richardson, Neal, Ian Cook, Nic Crane, Dewey Dunnington, Romain Fran<U+00E7>ois, Jonathan Keane, Drago<U+0219> Moldovan-Gr<U+00FC>nfeld, Jeroen Ooms, Jacob Wujciak-Jens, and Apache Arrow. 2024. Arrow: Integration to ’Apache’ ’Arrow’. https://CRAN.R-project.org/package=arrow.
Schauberger, Philipp, and Alexander Walker. 2023c. Openxlsx: Read, Write and Edit Xlsx Files. https://CRAN.R-project.org/package=openxlsx.
———. 2023a. Openxlsx: Read, Write and Edit Xlsx Files. https://CRAN.R-project.org/package=openxlsx.
———. 2023b. Openxlsx: Read, Write and Edit Xlsx Files. https://CRAN.R-project.org/package=openxlsx.
Tay, J. Kenneth, Balasubramanian Narasimhan, and Trevor Hastie. 2023. “Elastic Net Regularization Paths for All Generalized Linear Models.” Journal of Statistical Software 106 (1): 1–31. https://doi.org/10.18637/jss.v106.i01.
Whitehead, John. 1980. “Fitting Cox’s Regression Model to Survival Data Using GLIM.” Journal of the Royal Statistical Society. Series C (Applied Statistics) 29 (3): 268–75. http://www.jstor.org/stable/2346901.
Wickham, Hadley. 2016. Ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New York. https://ggplot2.tidyverse.org.
Wickham, Hadley, Mara Averick, Jennifer Bryan, Winston Chang, Lucy D’Agostino McGowan, Romain Fran<U+00E7>ois, Garrett Grolemund, et al. 2019. “Welcome to the tidyverse.” Journal of Open Source Software 4 (43): 1686. https://doi.org/10.21105/joss.01686.
Wickham, Hadley, Maximilian Girlich, Mark Fairbanks, and Ryan Dickerson. 2023. Dtplyr: Data Table Back-End for ’Dplyr’. https://CRAN.R-project.org/package=dtplyr.
Wilke, Claus O. 2024. Ggridges: Ridgeline Plots in ’Ggplot2’. https://CRAN.R-project.org/package=ggridges.