Vincent's Limit Breaks are as follows:
Lv. 1 Limit Break: Galian Beast (default)
Lv. 2 Limit Break: Death Gigas (kill 40 enemies while on Lv. 1 Limit)
Lv. 3 Limit Break: Hellmasker (kill 56 enemies while on Lv. 2 Limit)
You can obtain his final limit break, Chaos, the moment you get the submarine. You'll want to scan the underwater walls between the Nibel area and Gold Saucer. You'll find a crack that will take you to a dead end. Float up and you'll find a small cave on a tiny bit of land you can enter. Enter it with a Vincent in your party to get a small scene. Afterward, leave and fight 10 battles(the land outside the cave doesn't have random encounters) and return. You'll find his final limit which will lead to your trophy after it's learned.