AI’s Attempt at New Yorker Cartoons

Artificial Intelligence (AI) has made remarkable strides in generating jokes, but the burning question remains: Can AI truly comprehend humor? In this exploration, we delve into the scientific challenges posed to AI models through tasks derived from the New Yorker Cartoon Caption Contest, aiming to uncover the depth of AI’s understanding of humor.

The Experiment: Testing AI’s Wit

Task 1: Matching Jokes to Cartoons

The first challenge involved matching jokes to cartoons, a task requiring a grasp of the intricate relationships between visual elements and textual humor. The study included both multimodal models, which directly analyzed cartoon images, and language-only models, which relied on descriptions of visual scenes to simulate human-level visual understanding.

Task 2: Identifying Winning Captions

The second task tasked AI models and human participants with identifying winning captions from a pool of entries. Humans consistently outperformed AI across these tasks.

Task 3: Explaining the Humor

The pinnacle of the experiment was explaining why a winning caption was funny. Here, they explored the depth of understanding required to decipher the often indirect and playful allusions to human experience and culture present in New Yorker cartoons.

Results: AI’s Struggle with the Punchline

Findings revealed that both multimodal and language-only models struggled across all three tasks. Even the best multimodal models lagged 30 accuracy points behind human performance in matching jokes to cartoons. Moreover, when provided with ground-truth visual scene descriptors, human-authored explanations consistently outshone the best machine-authored ones, showcasing the existing gap in humor comprehension.

Implications and Future Possibilities: The Role of AI in Humor Creation

While machines may not fully “understand” humor, they display impressive capabilities. The question of true understanding remains debatable, but AI’s proficiency in these tasks is undeniable. The study opens doors to the potential use of AI as a collaborative tool for humorists, offering a fresh perspective on ideation and creativity in the realm of humor.

References

Cornell University. (2023, July 31). That’s funny — but AI models don’t get the joke. ScienceDaily. https://www.sciencedaily.com/releases/2023/07/230731122233.htm

Hessel, J., Marasovic, A., Hwang, J. D., Lee, L., Da, J., Zellers, R., Mankoff, R., & Choi, Y. (2023). Do androids laugh at Electric Sheep? humor “understanding” benchmarks from the New Yorker Caption Contest. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). https://doi.org/10.18653/v1/2023.acl-long.41

More like this

full moon picture

Does a Full Moon effect our Sleep?

There are many myths surrounding the effects of a full moon on humans from higher rates of...

My Experiences Being a TA for a STEM Class...

  During your time at Georgia Tech, as a STEM major, one of the things that you might...

Treating U87 EGFP Glibolastoma Cells with Nickel Sulfate!

Part of the reward of being a Biomedical Engineer is the opportunity to explore fascinating and cutting-edge...