Even corporate data is riddled with errors... often systematic errors. Far better than, say, X, but dangerous nonetheless. I would not rely on people or machines to totally get things right ALL of the time.
Just finished a weeklong job looking at 10 million medical records, culled from 40 million. The data runs, using neural network routines at MIT, identified just over 20,000 corrupted records. There are certainly more. LLMs use neural networking to get the job done. As a QA/QC guy, I was specifically looking for them and specifically wanted them culled for further examination. An error rate around 0.002 (0.2% or 1 in 500) is far lower than typical in corporate or government data, but could have particular effect when searching for a rare phenomenon, as we have been on this project.
Look at Boeing's predicament right now. Airframe manufacture in theory has very tight specs (design standards to design-in quality) and more-than-typical inspection regimes. But lots of stuff on the factory floor is never documented in any corporate record, in any factory other than factories that are 100% automated.
Typical protection against that is close-at-hand management (management level high enough to instantly allocate significant funds to fix an observed shortcoming). The 737-Max9 airframes are made in St. Louis, and the final plane assembly is in Washington State. And where has Boeing top management been since 2001? Chicago. The door plugs were supposed to be secured with 4 high-strength bolts -- which are usually quite brittle. Drop them onto a hard floor and they may crack internally. A typical "fix" would be to have the floor below the plug-install area padded and STILL throw away dropped bolts. Or "cue-up" the bolts in the assembly. Every once in awhile a bad bolt would sneak through, but the other three would do the job. The extra pad costs MONEY. Not much, but just a little. And it requires LABOR, which is in short supply.
BTW, I shot an episode of Invisible for Oprah Winfrey Network years ago at the Tucson aircraft boneyard. Worth the trip.
Well said, as always. I was gently pointing to this at the end of the piece, but have no fears-I'm going for the full beatdown in this week's piece on the UK/Fuji Horizon Postal Service disaster.
Even corporate data is riddled with errors... often systematic errors. Far better than, say, X, but dangerous nonetheless. I would not rely on people or machines to totally get things right ALL of the time.
Just finished a weeklong job looking at 10 million medical records, culled from 40 million. The data runs, using neural network routines at MIT, identified just over 20,000 corrupted records. There are certainly more. LLMs use neural networking to get the job done. As a QA/QC guy, I was specifically looking for them and specifically wanted them culled for further examination. An error rate around 0.002 (0.2% or 1 in 500) is far lower than typical in corporate or government data, but could have particular effect when searching for a rare phenomenon, as we have been on this project.
Look at Boeing's predicament right now. Airframe manufacture in theory has very tight specs (design standards to design-in quality) and more-than-typical inspection regimes. But lots of stuff on the factory floor is never documented in any corporate record, in any factory other than factories that are 100% automated.
Typical protection against that is close-at-hand management (management level high enough to instantly allocate significant funds to fix an observed shortcoming). The 737-Max9 airframes are made in St. Louis, and the final plane assembly is in Washington State. And where has Boeing top management been since 2001? Chicago. The door plugs were supposed to be secured with 4 high-strength bolts -- which are usually quite brittle. Drop them onto a hard floor and they may crack internally. A typical "fix" would be to have the floor below the plug-install area padded and STILL throw away dropped bolts. Or "cue-up" the bolts in the assembly. Every once in awhile a bad bolt would sneak through, but the other three would do the job. The extra pad costs MONEY. Not much, but just a little. And it requires LABOR, which is in short supply.
BTW, I shot an episode of Invisible for Oprah Winfrey Network years ago at the Tucson aircraft boneyard. Worth the trip.
Well said, as always. I was gently pointing to this at the end of the piece, but have no fears-I'm going for the full beatdown in this week's piece on the UK/Fuji Horizon Postal Service disaster.