I wonder about The Little People

I wonder about Little People wanting to be called that.
If I were a dwarf, or midget, or Al Pacino or whatever, I'd much prefer being called a dwarf, midget, or Mr. Pacino, because it's a simple description of my physical state of being, whereas "little person" sounds like a judgment being passed.
Billy Barty was a midget. Congressmen Bob Ney and Walter Jones are little people.