Predictive modelling uses statistics to predict outcomes. It is important to recognize that models are only as good as the modeler, who is only as good as the available data. Modelling is essentially an exercise in collecting and combining available information and, to some extent, estimates by the modeler. Some, but not all, of the possible risks to successful models include the fact that history cannot always predict the future, the problem of unknown unknowns and adversarial defeat of an algorithm.
Currently, with the Covid-19 dominating public concern and the media, considerable attention has been given to the recurring revisions to the widely used University of Washington’s Institute for Health Metrics and Evaluation (IHME) model. The initial IHME model, that has driven much of the government’s strategy in addressing the Covid-19’s threat to the world’s health, has proven to have been widely inaccurate and subject to frequent, major revisions. To be fair, IHME was dependent on out-of-date information and was attempting to shed some light on a black swan event (see our blog posted on April 22, 2020). But, science, as defined by Wikipedia, is a systematic enterprise that builds and organizes knowledge in the form of testable explanations and predictions. It is constantly subject to new information and can never be thought of as stationary or settled.
That being said, the model that the U.S. government was relying on was simply unreliable.
Investors should carefully consider the potential for revisions when relying on models in formulating investment strategies.
All comments and suggestions are welcome.
Walter J. Kirchberger, CFA