Is it polite to say "Can I talk to you in private" when you have company like they do on TV?

I was taught it was rude to do that since for all your guests know you could be talking about THEM, but I've seen this happen on TV so often I'm wondering if it's actually acceptable... or if it's just one of those things that happens on TV.