The only ones haveing different opinions are the COD players. They are used to kicking. We are not....
But okay. If there would be a vote, then the vote should be for only BF4 players, so actually only XI members who play BF4 on a regular basis. Even you drop in only, what is it? Once a week/two weeks for half a round? We have the server for over 2 years now, and we always done it like this and noone ever has been complaining about it. You are prioritized in que, so the wait is only 2 to 5 mins (if the map is popular), so i really don`t see a problem here...
Like Iboomboom said; BF4 is a different type of game, everyone is playing a role in their squad working towards a goal, abruptly disrupting that sounds unfair. It actually IS unfair. I would never return to a server that would kick me while i have a good round.