And we were. Chokoji is a small temple in quite a remote location. Now there is a small hot spring next door. It was a family temple of a branch of the Ogasawara Clan who ruled the small domain here in the Muromachi period. I believe they were also connected to Kannabiji.
Koyo is the Japanese word that refers to the changing colors of the Autumn leaves, though I believe it originally referred specifically to the Japanese maple with its red leaves, it now applies to the fyll range of colors including of course the golden yellow of the ginkgo .