This project researches computational models to represent different elements in human knowledge, aiming to understand how their interaction with image data underlies human image understanding. The modeling outcomes inform algorithmic fusion of human knowledge with image content to create novel representations of image semantics as a result of human-machine synergy. This representation is knowledge-centered, capable of capturing tacit domain knowledge of human experts, while its multimodal nature correlates with the underlying experts’ knowledge-based processing. Such representations enable knowledge-based matching of images, which can significantly enhance key visual problem solving tasks, such as semantic image retrieval and image grouping/classification, particularly for specialized domains.