I can give you one example where something like that happened.
All of our services use credstash to store secrets. We use KMS in order to encrypt, decrypt the secrets and when the app loads we load all the secrets into memory.
There was a major change to cryptography that caused credstash to break and in turn break a large set of our services.
This PR [1] was submitted by one of the engineers to force the lower version to credstash and we opened the dependencies again to reload credstash on all services.
The process to track it down and fix it took about an hour and all services were building correctly again.
We make a very conscious choice to break those things in real time and deal with them rather than hiding them behind locked dependencies you simply never upgrade.
All of our services use credstash to store secrets. We use KMS in order to encrypt, decrypt the secrets and when the app loads we load all the secrets into memory.
There was a major change to cryptography that caused credstash to break and in turn break a large set of our services.
This PR [1] was submitted by one of the engineers to force the lower version to credstash and we opened the dependencies again to reload credstash on all services.
The process to track it down and fix it took about an hour and all services were building correctly again.
We make a very conscious choice to break those things in real time and deal with them rather than hiding them behind locked dependencies you simply never upgrade.
[1] https://github.com/fugue/credstash/pull/174