“What giants?” requested Sancho Panza.
“These you see over there,” replied his grasp, “with the lengthy arms; typically they’re virtually two leagues lengthy.”
“Look, your grace,” Sancho responded, “these issues that seem over there aren’t giants however windmills, and what appears like their arms are the sails which might be turned by the wind and make the grindstone transfer.”
“It appears clear to me,” replied Don Quixote, “that thou artwork not well-versed within the matter of adventures: these are giants; and if thou artwork afraid, transfer apart and begin to pray while I enter with them in fierce and unequal fight.”
— Miguel de Cervantes, Don Quixote, Half I, Chapter VIII [1]
Introduction
The scientific methodology is constructed round a easy concept: formulate a speculation, check it towards actuality, and resolve whether or not the proof helps it.
In knowledge science, we do that consistently. We run experiments, construct proofs of idea, examine fashions, validate efficiency, and ask whether or not an concept works earlier than investing additional in it. We now have grow to be remarkably good at testing hypotheses earlier than trusting them.
However what occurs after they work?
A speculation that survives an experiment can grow to be a mannequin. A mannequin that performs nicely can grow to be a system. A profitable system can grow to be a course of, and a course of repeated for lengthy sufficient can finally grow to be the best way issues are carried out. Someplace alongside that path, an attention-grabbing reversal can happen: what as soon as needed to show itself towards actuality can finally grow to be a part of the lens by which we interpret it.
And actuality doesn’t stand nonetheless.
Populations change, and as a consequence, data-generating processes change with them. Applied sciences evolve, organizations adapt, and but previous success gives a robust motive to maintain trusting the assumptions that produced it.
This raises a query that goes past mannequin monitoring or technical efficiency:
How usually can we retest the assumptions behind one thing that also appears to work?
On this article, I need to discover that query at completely different ranges, d to knowledge acience strategies, and take into account a deceptively easy risk:
What if we preserve adapting our fashions and strategies with out adapting the best way we perceive the issue?
Kodak and Moody’s illustrate two completely different types of the identical underlying downside. Kodak represents the extra acquainted story: a corporation struggles to maneuver past the logic that made it profitable. Moody’s presents a extra delicate (and maybe extra harmful) model. The group did adapt. Fashions have been revised, methodologies developed, and new info was integrated as markets modified. But these adjustments might stay bounded by the assumptions already embedded within the system [2]. Kodak asks what occurs once we fail to alter. Moody’s asks a more durable query: what if we’re altering on a regular basis, however not altering what actually issues?
When Success Turns into a Constraint
Everyone is aware of the cautionary story of Kodak, the traditional case examine of an organization that didn’t adapt to alter. However I need to use Kodak to make some extent that goes past the standard story of technological disruption.
For many years, Kodak constructed a very profitable enterprise round a specific understanding of pictures: cameras generated demand for movie, movie generated recurring income, and processing accomplished the ecosystem. Digital pictures didn’t merely introduce a brand new expertise. It challenged the assumptions that made that system work [5].
Expertise is effective exactly as a result of it permits us to acknowledge patterns and make choices with out rediscovering all the things from scratch. However there’s a paradox: the extra profitable a specific interpretation of the world turns into, the better it’s to cease seeing it as an interpretation in any respect. This may be bolstered by what behavioral idea calls establishment bias: as soon as a specific means of working is established, we grow to be disproportionately inclined to protect it somewhat than rethink the options. Previous success makes that tendency even simpler to justify.
The identical factor can occur in knowledge science.
Our fashions don’t start with algorithms. They start with choices about how an issue must be represented. We resolve what knowledge issues, the way it must be reworked, which relationships are value modelling, what success appears like, and which assumptions are affordable sufficient to proceed.
At first, these are selections. However after they work repeatedly, they grow to be practices. Practices grow to be processes, and processes finally grow to be methodology. What started as a speculation about tips on how to remedy an issue can quietly grow to be the accepted means of fixing it.
And this creates a extra delicate sort of danger. The issue is now not merely whether or not a mannequin turns into outdated. It’s whether or not we are able to preserve updating the mannequin whereas leaving the best way we body the issue largely untouched.
Fashions Seize Actuality Via Assumptions
A mannequin learns from the world it has seen. The tough query is whether or not it stays helpful when the world adjustments.
The query is just not solely tips on how to predict Drug B, however why relationships discovered from Drug A ought to nonetheless maintain for it.
An identical picture as above was proven to me on the finish of an interview. With about 5 minutes left and virtually no context, the interviewer requested me to clarify what I used to be seeing. The precise slide was barely completely different, however the issue was primarily the identical.
A number of questions instantly got here to thoughts, however one stood out: if we study from Drug A, what makes us assured that the relationships we discovered will nonetheless maintain for Drug B?
You’ve in all probability heard this many occasions: a predictive mannequin is, by definition, a simplification. It learns relationships from observations generated below specific situations. We select variables, outline outcomes, make assumptions, and scale back a fancy actuality into one thing we are able to mannequin.
There may be nothing inherently mistaken with utilizing historic medication to foretell the uptake of a brand new one [3, 4]. In actual fact, studying from associated populations, domains, or duties is a well-established concept in statistics and machine studying. However doing so requires assumptions about what’s transferable. Are the related populations comparable? Are the mechanisms driving uptake sufficiently secure? Do the predictors have the identical which means? Has the market or data-generating course of modified in a means that breaks the relationships discovered traditionally?
If these assumptions are specific and we’ve got proof that they moderately maintain, utilizing Drug A to find out about Drug B could also be fully justified. The issue is assuming transportability somewhat than establishing it [4].
And that was exactly what made the interview query tough. With the data obtainable on the slide, I might perceive the proposed modelling technique, however I couldn’t conclude that making use of relationships discovered from historic medication to Drug B was essentially the correct strategy. Crucial info was not solely the mannequin or its historic efficiency. It was whether or not the assumptions that allowed us to maneuver from A to B have been defensible.
As soon as Drug B launches and its precise uptake turns into observable, these assumptions could be confronted with genuinely exterior proof. Till then, historic efficiency tells us what labored on the earth we noticed, not routinely what is going to work on the earth we are attempting to foretell.
A mannequin learns from the world it has seen. The tough query is what should stay true for it to work on the earth it has not.
And this results in a deeper downside. We all know that transferring a mannequin into a brand new surroundings ought to drive us to query what makes that switch legitimate. However what if, as an alternative, we preserve modifying a mannequin whereas the assumptions defining the issue grow to be more and more tough to see, not to mention problem?
That’s the place Moody’s turns into significantly attention-grabbing.
Moody’s: The Phantasm of Adaptation
Moody’s gives a way more concrete instance of what occurs when the world adjustments quicker than the assumptions used to mannequin it.
Moody’s is among the main credit standing companies. A part of its job is to evaluate how dangerous monetary merchandise are and translate that danger into rankings comparable to AAA. Within the years main as much as the 2008 monetary disaster, Moody’s rated securities backed by hundreds of residential mortgages, which means that a part of its job was exactly to evaluate the credit score danger embedded in these merchandise.
The issue is sort of acquainted: given the traits of a pool of mortgages and the debtors behind them, estimate how that pool will carry out below completely different financial situations.
To estimate their danger, Moody’s developed fashions that used details about particular person debtors and mortgages, along with historic knowledge and simulations of financial situations, to estimate how a lot cash a pool of mortgages might lose.
So, the method was extremely model-driven. Moody’s obtained a loan-level dataset containing details about the person mortgages in a pool. Analysts fed this info right into a proprietary mannequin that estimated anticipated losses and the way a lot safety could be required for the safety to realize a specific score. These outputs then knowledgeable analysts and score committees earlier than a last score was printed.
Then the mortgage market modified dramatically…
Subprime lending expanded, lending requirements grew to become a lot looser, more and more complicated securities appeared, and mortgages requiring little or no documentation grew to become way more frequent. The inhabitants Moody’s was making an attempt to mannequin was now not the identical.
And Moody’s seen.
It developed a extra refined mannequin (known as M3) that simulated mortgage efficiency below completely different financial situations. As subprime lending exploded, it went additional and developed M3 Subprime, particularly calibrated for this new sort of mortgage [2].
However the assumptions underlying the mannequin modified far much less.
Moody’s continued to rely closely on historic relationships to simulate future financial situations. The macroeconomic simulation engine used for M3 was carried into M3 Subprime. The corporate continued to belief info provided concerning the underlying loans whilst lending requirements and knowledge high quality deteriorated. And since there was restricted historic proof for this quickly altering subprime market, some parameters needed to rely closely on knowledgeable judgement [2].
In abstract, Moody’s improved the mannequin, added complexity, recalibrated parameters, and created a model particularly for subprime mortgages. What it didn’t totally do was rebuild its illustration of the issue across the risk that the market itself had basically modified.
When the housing market collapsed, the assumptions embedded in that illustration now not described the world the mannequin was being requested to foretell.
Strategies Can Turn out to be Fashions Too
The identical reasoning could be utilized past particular person fashions.
Information science itself has accrued strategies in response to actual issues. The scientific methodology offers us a framework for formulating hypotheses, testing them towards proof, and revising them after they fail. Software program engineering provides reproducibility, testing, and maintainability. DevOps and MLOps tackle deployment, monitoring, versioning, and steady operation. Governance provides controls for more and more consequential methods.
These practices didn’t seem arbitrarily. They’re accrued options to issues we encountered alongside the best way. And that’s exactly why they deserve the identical scrutiny as fashions.
A choice that works turns into a observe. A observe turns into a course of. The method acquires instruments, roles, KPIs, documentation, and governance. Ultimately, we might grow to be superb at executing it with out remembering which assumptions made it applicable within the first place.
This doesn’t imply consistently reinventing how we work. That might defeat the aim of accrued expertise. It means recognizing that strategies, like fashions, have situations below which they make sense.
We routinely ask whether or not knowledge has drifted, whether or not a mannequin wants retraining, or whether or not its efficiency has deteriorated. Maybe we should always often ask the equal query about our strategies:
What would wish to alter on the earth for this fashion of working to cease making sense?
Conclusion
We spend an excessive amount of effort validating concepts earlier than we belief them. Maybe we should always spend extra time revalidating the assumptions that made them profitable within the first place.
Kodak reminds us that success could make the assumptions behind a means of working more and more tough to see. Analogue-based pre-launch forecasting makes the position of these assumptions specific: transferring patterns from historic merchandise is legitimate solely insofar because the situations that make them comparable nonetheless maintain. Moody’s then reveals the extra delicate downside: a system can proceed to adapt whereas leaving exactly these underlying assumptions largely untouched.
A mannequin could be monitored, retrained, and even improved whereas the best way the issue itself is framed stays largely unchanged. Monitoring can inform us when efficiency deteriorates. Clarification and uncertainty might help us perceive when a prediction deserves much less confidence. However none of those, by themselves, tells us whether or not the assumptions defining the issue nonetheless make sense.
Maybe some of the necessary properties of a very good mannequin is just not solely realizing what it predicts, however realizing the place it stops and the place questioning ought to start.
The lesson is just not that have, fashions, or established strategies must be distrusted. It’s virtually the other. They’re worthwhile as a result of they encode what we’ve got discovered. However accrued data ought to stay open to proof, views, and questions that come from outdoors the body through which it was constructed.
What we’ve got discovered ought to by no means grow to be indistinguishable from what should be true.
Evolution is dependent upon conserving our assumptions open to problem, particularly when success has made them tough to see. That’s a part of good scientific observe, however it’s hardly ever a straightforward one.
References
[1] Cervantes Saavedra, M. de. (1605). Don Quixote of La Mancha, Quantity I, Half One, Chapter VIII. English translation within the Publiconsulting Media on-line version. Publiconsulting Media. Chapter VIII — Don Quixote of La Mancha
[2] Omidvar, O., Safavi, M., & Glaser, V. L. (2023). Algorithmic routines and dynamic inertia: How organizations avoid adapting to changes in the environment. Journal of Administration Research, 60(2), 313-345.
[3] Robey, S. H., & David, F. S. (2017). Drug launch curves within the fashionable period. Nature Opinions Drug Discovery, 16, 13–14. DOI: 10.1038/nrd.2016.236.
[4] Guseo, R., Dalla Valle, A., Furlan, C., Guidolin, M., & Mortarino, C. (2017). Pre-launch forecasting of a pharmaceutical drug. Worldwide Journal of Pharmaceutical and Healthcare Advertising, 11(4), 412–438. DOI: 10.1108/IJPHM-07-2016-0036.
[5] Lucas Jr, H. C., & Goh, J. M. (2009). Disruptive expertise: How Kodak missed the digital pictures revolution. The Journal of Strategic Data Techniques, 18(1), 46-55.
[6] Morrison, E. W. (2023). Worker voice and silence: Taking inventory a decade later. Annual evaluation of organizational psychology and organizational habits, 10(1), 79-107.
