There are 5 memories for the Old Man. The first 4 memories must be collected the first time you get control of him. The final memory can only be collected when you're forced to enter the Old Man's house.
(22:23) - Memory 1
The next 3 Memories are found in the cemetery.
(22:50) - Memory 2
(23:29) - Memory 3
(23:49) - Memory 4
Keep playing as the Old Man until you're forced to enter his house.
(24:11) - Memory 5
*** Spoiler - click to reveal ***When you have to enter the Old Man's house, walk out the back door and turn left.
head around the side of the house to find the final memory.