As usual for me, I try to visit as many of the local shrines as I can, and in Arita the most popular is the Tozan Shrine featuring porcelain komainu and torii. You will also see Ebisu statues along the local roads as Hizen, the former name for Nagasaki and Saga prefectures, the Ebisu cult was very popular.
There are no particularly interesting temples in the town, though one does have some nice nio.