In order to defeat Timid Swimmer you need to get him to hit the big red bollards, this will cause them to break and you can jump on them and attack him (with the quick time events).
Simply navigate to the highest one first, stand behind it and he will swing and break it, follow the button prompts and after 3 times you will have defeated him!