Well, better than the other way around:
Code works perfectly on your test device but breaks down when deployed to field devices with slightly different timing characteristics or whatever.
Bonus points if it only occurs every few weeks, preferably at night shift and crashes a whole production line… 🫣
(Incident totally fictitious, definitely no people out of this thread involved, just move on, really nothing to see here!)
I’m currently hunting a bug that happens like every 1000 iteration of the thing happening.
Like, I’m telling the hardware to do something and it works pretty much all the time, but over the day, the errors add up
I have no clue why it happens, but can’t really turn up the debug logs that much, because with so many things happening, I’d produce like a shitload of data.
But I can’t really narrow it down otherwise
And it seems we’re in the same kind of shit business ;-)
Real time processes and automation, with customers having problems at night shift, because the maintenance guys during that shift are usually not as good - or it’s just bad luck
At one of my last business trips I was already at the airport on my happily way home, when I’ve got call.
Needed to get my luggage back, new rental car and get a place at the hotel.
Just to discover, that after 15 years the hardware acted up in a way it never did before.
At least I could now include a warning message, if this weird situation ever happens again, but that was a tough one to swallow…
Best things are, when you throw in some debug output and that changes the timings just as much to not make the bug happen anymore
I’ve lost more hair to that, than my age…
Well, better than the other way around:
Code works perfectly on your test device but breaks down when deployed to field devices with slightly different timing characteristics or whatever.
Bonus points if it only occurs every few weeks, preferably at night shift and crashes a whole production line… 🫣
(Incident totally fictitious, definitely no people out of this thread involved, just move on, really nothing to see here!)
True that…
Happens too fucking often as well
I’m currently hunting a bug that happens like every 1000 iteration of the thing happening.
Like, I’m telling the hardware to do something and it works pretty much all the time, but over the day, the errors add up
I have no clue why it happens, but can’t really turn up the debug logs that much, because with so many things happening, I’d produce like a shitload of data.
But I can’t really narrow it down otherwise
And it seems we’re in the same kind of shit business ;-)
Real time processes and automation, with customers having problems at night shift, because the maintenance guys during that shift are usually not as good - or it’s just bad luck
At one of my last business trips I was already at the airport on my happily way home, when I’ve got call.
Needed to get my luggage back, new rental car and get a place at the hotel.
Just to discover, that after 15 years the hardware acted up in a way it never did before.
At least I could now include a warning message, if this weird situation ever happens again, but that was a tough one to swallow…