- Tic-tac-toe memory bomb Evolved player makes invalid moves far away in the board, causing opponent players to run out of memory and crash
- Ruler detector AI trained to classify skin lesions as potentially cancerous learns that lesions photographed next to a ruler are more likely to be malignant.
I'm laughing and crying. In an artificial life simulation where survival required energy but giving birth had no energy cost, one species evolved a sedentary lifestyle that consisted mostly of mating in order to produce new children which could be eaten (or used as mates to produce more edible children).
Evaluation metric: “the output of sort is in sorted order” Solution: “always output the empty set” The AI version of YAGNI in Test Driven Developement ... write just enough code to pass the test. Sure, but you're supposed to write more tests! When repairing a sorting program, genetic debugging algorithm GenProg made it output an empty list, which was considered a sorted list by the evaluation metric.
First real customer walks in and asks where the bathroom is. The bar bursts into flames, killing everyone.QA tester walks into a bar. He orders a beer. Orders 0 beers. Orders 99999999999 beers. Orders a lizard. Orders -1 beers. Orders a ueicbksjdhd.
As someone who builds software for a living, it's interesting to see how this is an issue for AI in exactly the same way it's an issue for software teams. If you just build what you are asked for, it will be the wrong thing. And even if you're careful, you will learn what people actually want only from their disappointed response to you actually building something. This is why we build things in small chunks and get feedback along the way: people are not good at converting their imaginative vision into written, spoken or encoded instructions, and there will always be something silently assumed. The AIs have it even harder than the human teams though, since we humans are (for the time being) better at predicting common human oversights and reading between the lines of the instructions. Perhaps one day we'll be able to preprocess our instructions through a "figure out what the human probably meant to say" AI.
Frankly, it's the thing that drives me the craziest about machinists. Engineers are like gods to them - stupid, vengeful gods that aren't to be respected because obviously they don't know what the fuck they're doing but boy howdy thou shalt not question the holy writ on the drawing or the hand of the Almighty will rise up and smite thee! All last quarter was an adventure in "interpret this poorly-dimensioned, nonsensical part made out of inappropriate material" when any engineer that had had so much as one part built would have learned that you always want to make things as absolutely legible as possible because interpolation is always going to be suboptimal. And I can't recall a single device or component I ever designed that did not go through at least a few rounds of "so, guys, how hard is this going to be to build?" It isn't even about a lack of vision - it's about a lack of comprehension of someone else's craft. The decent craftsmen know this and account for it. "Does this need to be solid stainless? And how critical is this dimension? Because I can buy a brass fitting from Home Depot and bore it out a bit and you'll be within fifteen thousandths. And then you can have it this afternoon instead of Q3 because the 5-axis is making DoD shit until those shelves are full." But unless you have clever humans on both sides of the equation, stupid mistakes happen. I was working in a small town that happened to be an hour drive from my home in a suburb of Seattle. I could get socks delivered to my home by Amazon within 24 hours. I could get socks delivered to my small town within 6 weeks. Amazon had a bunch of skookum algorithms to figure out how to ship what where, but it didn't have any "use alternate shipping methods if there's a six week difference in a sixty minute drive" patches to prevent them from losing the sale. 'cuz there was also a Walmart half a mile away... and guaranteed, that's what most people did.
There are corners of Amazon that are the result of clever people squeezing the numbers as hard as they can and turning coal into diamonds (and making their hands achey and black). There are other corners of Amazon that are the result of Jeff Bezos demanding things. There's a bit in The Everything Store where they're talking about the design of the Kindle. Super-secret, nobody talk about it, rootin'tootin'hifalootin' designers from Germany and it's this beautiful e-ink thing that's light and thin and glorious and Jeff Bezos hates it because it doesn't have a keyboard. Well of course it doesn't have a keyboard, asshole, it's a reader, said the designers but no, Bezos insisted that customers needed to be able to buy Kindle books from a goddamn Kindle and to do that they either needed a touch screen which didn't work with the refresh rates or they needed a fucking keyboard and 3G and he was such a dick about it that the designers walked and the world got
Was gonna post this too! Feel a tinge of patriotism for the genetic algorithm that discovered fierljeppen. Mah people.