It works with a POT becasue it's just a Resistor, polarity doesn't technically matter, but swapping the power/ground to it does invert the output. With a Hall sensor, polarity matters because they have electronics in them, so swapping the power/ground to them is bound to cause all kinds of issues and/or damage something.
You could have pulled that kind of thing off on the older DS3 style Hall sticks that had 4 pins on them, but that's going to fly on any 3 pin style.