To benchmark progress in visual relationship detection, we also introduce a new dataset containing 5000 images with 37,993 thousand relationships. THe dataset contains 100 object categories and 70 predicate categories connecting those objects together. Predicates can be widely categorized into the 5 following types:

Example Results

What the predictions from the model looks like.
Browse more results on using our demo using our github code.

Content based Image Retrieval

By understanding the relationships between objects in images, we show that our model can outperform existing image retrieval models that use SIFT, CNN or Visual Phrase features.

Bibtex

@inproceedings{lu2016visual,
  title={Visual Relationship Detection with Language Priors},
  author={Lu, Cewu and Krishna, Ranjay and Bernstein, Michael and Fei-Fei, Li},
  booktitle={European Conference on Computer Vision},
  year={2016}
  }