There are several caves. All of them are carved into sandstone. The main cave is in Gupta architecture style. It is believed that these rock-cut monastery cells were originally used by Jain monks, the basis of which is their architectural styles on the pattern of Udayagiri and Khandegiri.
The largest and most popular cave has a huge monolith of Lord Vishnu in the reclining posture. This monolith is made from a single granite block and is located on the second storey of the temple, which has a total of four stories. The huge statue of Lord Vishnu is in a reclining posture.
Other shrines inside the cave are dedicated to Trimurti: to Brahma, Vishnu and Shiva. Main cave belongs to the earliest examples of Gupta architecture, primarily primitive rock-cut monastery cells carved into the sandstone hills. Initially caves were shaped as a Buddhist monastery and the first floor still retains style of Buddhist vihara, including some Buddhist sculptures. The walls of the caves display sculptures carved by skilled craftsmen.
Undavalli caves are associated with the Vishnukundina kings of 420 to 620 A.D. They are dedicated to Anantapadmanabha Swamy and Narisimha Swamy. Caves are surrounded by the green countryside. From the high hill above the cave overlooking the Krishna river many fine specimens of rock cut Hindu architecture can be seen.