The Astrologer dwells with his friend at the Lighthouse southeast of the village and only comes around every Sloth Day (the blue moon). At first, he will request of you a skull, which will get you another trophy, but if you want to leave this place, you'd do well to help him with his own dilemma.