The description of image semantic information is a fundamental task for realizing the intelligent retrieval and knowledge discovery of Thangka. On the basis of introducing the basic principles of image semantic information description, according to the composition characteristics of Thangka images, this paper constructs a conceptual framework for describing the semantic information of Thangka images, and explores the structure and implementation ideas of the semantic metadata model of Thangka images. It is believed that the semantic information of Thangka should be considered from three levels: underlying feature information, object semantic information, and conceptual semantic information. By combining the attributes and relationships of different objects, the corresponding metadata model is designed, and the semantic information is catalogued with reference to the domain ontology.