How good is the model?

Evaluating the player aging model

Previously, we fit a mixed effects model for player ops including

• splines (to capture the ascent, peak, and descent)
• random effects (to capture the variety in player performance)

Let’s call it the “PAMS model” for “player aging mixed model with splines”.

Let’s evaluate its accuracy and benchmark against some similar, non-statistical approaches to forecasting player OPS.

lme4 model summary

Linear mixed model fit by REML ['lmerMod']
Formula: ops ~ (centered_age | name) + bs(centered_age, df = 3)
   Data: filter(hit, season < 2024)
REML criterion at convergence: -9209.238
Random effects:
 Groups   Name         Std.Dev. Corr
 name     (Intercept)  0.066227     
          centered_age 0.005827 0.05
 Residual              0.072664     
Number of obs: 4381, groups:  name, 567
Fixed Effects:
              (Intercept)  bs(centered_age, df = 3)1  
                  0.72236                    0.13814  
bs(centered_age, df = 3)2  bs(centered_age, df = 3)3  
                  0.01667                   -0.20310  

Test data

We will fit the model on data from 2005 - 2023 and test it it on 2024.

Benchmarks

We will compare the model to three simple benchmarks.

Table

Quality of fit for the statistical model and three benchmarks