As a tech enthusiast, this doesn’t surprise me one bit; one of my most used repair techniques is to do nothing and wait. A lot of problems just go away by themselves. But I’m still very curious how the 16th reboot was fixing that bluescreen.
I think the reasoning was that there was a race condition between the code causing the bluescreen and the code updating to avoid the bluescreen so rebooting 15 times would give a lot of opportunities for the updater to win the race.
But if it was a race condition, then some computers would just boot normally. I didn’t see anyone report that the issue was happening selectively. And that wouldn’t even be fix, just a one-off boot. Unless the file is removed the issue will come back on next reboot.
It’s probably one central server controlling access to the network or distributing images or something. So they need to reboot one machine in that cluster enough times and all of the machines in the cluster will work.
The vulnerability broke every machine, the fix was the one that took multiple reboots to apply.
I’m not sure we are talking about the same issue. In case of CrowdStrike, the update pushed a botched file that crashed the kernel on boot. Until the file was removed, the machine wouldn’t even boot to be patched.
I’m saying that in production, the screens and whatnot probably aren’t fetching that file on boot, they’re probably pulling from some central server. So in the case of an airport, each of those screens is probably pulling images from a local server over PXE, and the server pulls the updates from CrowdStrike. So once you get the server and images patched, you just power cycle all of the devices on the network and they’re fixed.
So the impact would be a handful of servers in a local server rack, and then remote power cycle. If they’re using POE kiosks (which they should be using), it’s just a simple call to each of the switches to force them to re-PXE boot and pull down a new image. So you won’t see IT people running around the airport, they’ll be in the server room cycling servers and then sending power-cycle commands to each region of the airport.
As a tech enthusiast, this doesn’t surprise me one bit; one of my most used repair techniques is to do nothing and wait. A lot of problems just go away by themselves. But I’m still very curious how the 16th reboot was fixing that bluescreen.
I think the reasoning was that there was a race condition between the code causing the bluescreen and the code updating to avoid the bluescreen so rebooting 15 times would give a lot of opportunities for the updater to win the race.
But if it was a race condition, then some computers would just boot normally. I didn’t see anyone report that the issue was happening selectively. And that wouldn’t even be fix, just a one-off boot. Unless the file is removed the issue will come back on next reboot.
Your server also had to be patched for that to work I think.
It’s probably one central server controlling access to the network or distributing images or something. So they need to reboot one machine in that cluster enough times and all of the machines in the cluster will work.
The vulnerability broke every machine, the fix was the one that took multiple reboots to apply.
I’m not sure we are talking about the same issue. In case of CrowdStrike, the update pushed a botched file that crashed the kernel on boot. Until the file was removed, the machine wouldn’t even boot to be patched.
Yes, that’s what I’m talking about.
I’m saying that in production, the screens and whatnot probably aren’t fetching that file on boot, they’re probably pulling from some central server. So in the case of an airport, each of those screens is probably pulling images from a local server over PXE, and the server pulls the updates from CrowdStrike. So once you get the server and images patched, you just power cycle all of the devices on the network and they’re fixed.
So the impact would be a handful of servers in a local server rack, and then remote power cycle. If they’re using POE kiosks (which they should be using), it’s just a simple call to each of the switches to force them to re-PXE boot and pull down a new image. So you won’t see IT people running around the airport, they’ll be in the server room cycling servers and then sending power-cycle commands to each region of the airport.
The trick is to sleep, have an issue? Take a nap and come back to find it working somehow