You need to jump onto the large pillars and climb to the top of them. Once here, shoot an arrow at the Colossus and he will ram the pillar. Once he does this 2 times, it will fall. You need to follow these large pillars until it circles back to the beginning.
To take his armour off you need to get him to hit the support beam for the platform. Get him to hit this, and it will fall on him revealing his weak spot. Get on his back and stab until he is defeated.