Can an image be partisan? (AI says Yes!)

It seems like almost every political Facebook ad contains a picture of Donald Trump. Trump’s campaign shows him, smiling, with a thumbs-up or hands raised in victory. Republicans show themselves with him. And Democrats show Trump in monochrome, looking sinister.

A person could probably take a look at one of these images—stripped of the rest of the ad—and make a good guess at which party the image came from.

Could an AI model do the same?

We tried and … it worked. Our model got 93% accuracy predicting whether an image in an ad came from a Republican, Democratic, or non-partisan political ad.

How does this work?

This is a machine-learning model, so it learns by example. I gave it about 500 images each for the three classes of political ad and it learned what it could.

The model uses all sorts of fancy math to find patterns in the images that it can use to distinguish between the classes. We can hope and hypothesize that it’s picking up on the same sorts of things that we might guess would distinguish different kinds of ads — like color and facial expression. But existing tools don’t make this easy to verify.

What is this good for?

Not much!

If we’re trying to figure out the political viewpoint of an ad, the text is probably a better indicator than the image.

What is it bad for?

A lot!

A key problem with AI “solutions” is that they’re trained on data from one narrow domain, but used in a broader one. If you submit ads to this model that aren’t from a political Facebook ad, who knows what it will guess! It hasn’t been trained on those images, so it doesn’t know what to do.

It’s also probably pretty bad on ads from local races, where the tropes from national and state-level races may not filter down.

Give it a try

You can try it out yourself here.

To start with, take a screenshot or download the image from these three ads and give it a whirl.

Then try with images from other ads … or other non-ad pictures, like this picture of my cat.

a picture of Jeremy's cat

She is non-partisan, as the model confirms.

It doesn’t work perfectly, as with this Trump ad, which the model is convinced is from a Democrat, perhaps because of Trump’s smirk or the funny lighting.

The model, which was trained with Fast.ai, has been converted into an ONNX model, which runs in JavaScript. So the model runs entirely on your computer and images aren’t uploaded anywhere. We used this awesome tutorial to convert the model.