Electrophysiology Setup From The Lab I Used To Work In, Now In Painting Form!

Electrophysiology setup from the lab I used to work in, now in painting form!

Do neural nets dream of electric sheep?

If you’ve been on the internet today, you’ve probably interacted with a neural network. They’re a type of machine learning algorithm that’s used for everything from language translation to finance modeling. One of their specialties is image recognition. Several companies - including Google, Microsoft, IBM, and Facebook - have their own algorithms for labeling photos. But image recognition algorithms can make really bizarre mistakes.

Microsoft Azure’s computer vision API added the above caption and tags. But there are no sheep in the image of above. None. I zoomed all the way in and inspected every speck. 

It also tagged sheep in this image. I happen to know there were sheep nearby. But none actually present.

Here’s one more example. In fact, the neural network hallucinated sheep every time it saw a landscape of this type. What’s going on here?

The way neural networks learn is by looking at lots of examples. In this case, its trainers gave it lots of images that humans had labeled by hand - and lots of those images contained sheep. Starting with no knowledge at all of what it was seeing, the neural network had to make up rules about which images should be labeled “sheep”. And it looks like it hasn’t realized that “sheep” means the actual animal, not just a sort of treeless grassiness. (Similarly, it labeled the second image with “rainbow” likely because it was wet and rainy, not realizing that the band of colors is essential).

Are neural networks just hyper-vigilant, finding sheep everywhere? No, as it turns out. They only see sheep where they expect to see them. They can find sheep easily in fields and mountainsides, but as soon as sheep start showing up in weird places, it becomes obvious how much the algorithms rely on guessing and probabilities.

Bring sheep indoors, and they’re labeled as cats. Pick up a sheep (or a goat) in your arms, and they’re labeled as dogs.

Paint them orange, and they become flowers.

Put the sheep on leashes, and they’re labeled as dogs. Put them in cars, and they’re dogs or cats. If they’re in the water, they could end up being labeled as birds or even polar bears.

And if goats climb trees, they become birds. Or possibly giraffes. (It turns out that Microsoft Azure is somewhat notorious for seeing giraffes everywhere due to a rumored overabundance of giraffes in the original dataset)

The thing is, neural networks match patterns. They see patches of furlike texture, a bunch of green, and conclude that there are sheep. If they see fur and kitchen shapes, it may conclude instead that there are cats.

If life plays by the rules, image recognition works well. But as soon as people - or sheep - do something unexpected, the algorithms show their weaknesses.

Want to sneak something past a neural network? In a delightfully cyberpunk twist, surrealism might be the answer. Maybe future secret agents will dress in chicken costumes, or drive cow-spotted cars.

There are lots, lots more examples of hilarious mistakes in a Twitter thread I started with the simple question:

And you can test Microsoft Azure’s image recognition API and see for yourself that even top-notch algorithms are relying on probability and luck. Another algorithm, NeuralTalk2, is the one I mostly used for the Twitter thread.

Want to know when I post another experiment? You can sign up here.

To those who are losing a lot of their will to create in the wake of President Tyrant: Don’t let him and his orcs win. People NEED your books, stories, poems, paintings, jewelry, dolls, knitting, tapestries, vases, weaving, dishes, every creation that comes from your hands. Every creation is a punch back at the haters and the heartless. Every word puts hope or thought or dreams or solace or fire into those who read it. You become a different voice from the bullyraggers and the foolish; your ideals, wishes and convictions reach your audience, whether they are reading Dr. Seuss or James Joyce. You convey food for hope and imagination whether you realize it or not, and the most innocuous-seeming work gives those who partake of it something to go on with. Keep soldiering on. Comedian or philosopher, baker or glassblower, writer of tomes or fan fiction, you’re needed now more than ever.

Tamora Pierce (via withaliensandrainbows)

Finally got to cleaning these kids up and putting them together!

Forgot to post this earlier so here it is!

Constantly torn between “I can’t ask for help bc then I’m annoying and everyone will be mad” and “I must ask for help at every possible stage because I might do it wrong and then everyone will be mad” ya feel