To successfully execute an alien, you have to stun them by weakening them enough so they sit there on their knees gazed for a few moments before they get back up again, walk up to them and press the
button to execute, Duke will kick or punch their head off. The ways I got to stun the aliens most of the time was to melee them up close with
until they go down, shoot at their torso a certain amount or throw something at them.
If you do not have this trophy by the mission "The Shrunk Machine"
, then in this mission some workers will give you a "Freeze ray"
. If you use this freeze way to freeze an enemy, you can walk up to them and do an easy execution, getting you towards the trophy for killing frozen aliens too.