In the grounds of Usuki castle was another small shrine, founded in 1879 after the castle had been dismantled, the size of the trees and the pond and landscaping certainly suggests something was here before that.
It is a Gokoku Shrine, basically a local version of the infamous Yasukuni Shrine that enshrines the spirits of those who died fighting for the emperor.
In pre-modern times the castle was the focus of political power, and once the castles were dismantled upon the creation of the modern state of Japan many of the castle ruins had Gokoku shrines built within them to give these new state-worshiping shrines legitimacy.
I suspect that there was a shrine here before but I may be wrong