The analog stick does not work the same way as the momentary switches used for the d-pad and other buttons on the PSP. It uses two potentiometers (for x and y axis) which allow the position of the stick to be determined by how much resistance is being placed on the current traveling through them. Because the stick always returns to the center, there is a default resistance applied just by it being connected, this means that joystick movements will drift to the corner when there isn't a proper connection, or when no joystick is connected at all.
*edit* I'm a slow typer, glad to see you solved the problem.
