Columbia University

Technology Ventures

Emotion intelligence for photo-sharing apps

Technology #cu14228

Questions about this technology? Ask a Technology Manager

Download Printable PDF

Image Gallery
Shih-fu Chang
Managed By
Satish Rao

Photo sharing has exploded as a social media activity for online users. On platforms such as Facebook, Twitter and Instagram, online users post images such as selfies, wedding pictures and situational photos to connect with their social network. A large motivation for photo sharing is eliciting response from one’s social network in the form of likes and comments. Commentary creates an opportunity for individuals and brands to connect with online users. This technology provides a powerful tool to deduce the sentiment evoked by a shared image and to offer an automated suitable response comment for social networking. This digital emotional intelligence could be useful for user-targeted advertising.

Infers sentiment from semantic analysis of photos and provides appropriate feedback

This technology analyzes a visual image to uncover the subject matter. It then utilizes a database of subject matter and emotion associations to deduce an appropriate comment based on the predicted emotion evoked. This technology is able to move beyond the semantics of an image to uncover the sentiment behind it. It does not require an image to have any text labeling or tagging to independently extract the subject matter or sentiment.

A prototype of the technology has been tested for a growing database of images, utilizing adjective noun pairs to deduce emotional state.

Lead Inventor:

Shih-Fu Chang, Ph.D.


  • Predictive comments for photo-sharing apps
  • Targeted advertising for brands
  • Emotionally intelligent customer relations for automated visual systems
  • Categorizing stock and search engine images
  • Public relations to simulate expected public response to visual content


  • Deduces sentiment from visual content without need of tagged keywords
  • Provides predictive emotional response

Patent information:

Patent Pending

Tech Ventures Reference: IR CU14228

Related Publications: