This sounds more difficult than it actually is if you have Rhin with you.
You only have to have the currently active party dislike you, so only three people. Be warned though that it doesn't count if one of the characters despises you but instantly leaves your party because of it. Happened to me with Aligern and it didn't work that way.
In the end I only had to work long-term on making one character dislike me. In my case that was Tybir who I treated pretty badly.
The other two were Rhin and Erritis. Both of them actually liked my main character, Rhin was at maximum affection for him and Erritis one only one level below.
Once you reach the bloom and after sleeping there for the first time an NPC called the Decanted will appear in front of the slave auction. He is interested in buying Rhin. Agree to sell her and Rhin's affection for you will immediately drop to despising you. Erritis will stop the deal from going through and also despise you instantly, but neither of them will leave your party. At least not if they liked you before, I can't tell if maybe they leave if their relation to the main character was less good before.
That's two characters you can instantly get from liking you a lot to despising you, leaving only one you have to work on for longer.