As is the case in Western India, in East India Buddhist caves were excavated mainly in the coastal area of Andhra Pradesh and Odisha. The Guntapalle in Eluru district is one of the main sites in Andhra Pradesh. The caves and the structured monasteries were excavated along the hills. Perhaps, it is one of the few sites where structures such as stupas, viharas, and caves are excavated at the same time.
Guntapalle chaitya cave is circular in shape with a stupa in the circular hall and a chaitya arch at the entrance. The cave is small compared to those in western India. In spite of their small dimensions, the main vihara caves have been decorated with chaitya arches on the exterior. They are rectangular with vaulted roofs and are one-story or two-story in design, with no central hall.
The excavations date back to the second century BCE, though they have been extended over the years, they all depict a vihara design.
Other than Guntapalle, Rampaerrampallam is another cave site that has moderately small excavations, but there are rock-cut stupas on the hillock. During the fourth and fifth centuries CE, caves were excavated near Vishakhapatnam, and a large rock-cut stupa was carved from the hillock. It has the largest rock-cut stupas in the country, making it unique. Numerous votive rock-cut stupas have been discovered all over the hillock.