Another thing you might want to check is the small rubber pad that sits between the analog stick and the mobo. If that is damaged even slightly then it will cause the joystick to drift or not work at all. Hopefully it should just be a case of tightening up the screw on the bottom left of the psp case, under the analog. If not, check the pad.
As socomhacker said, sand down the bottoms of the buttons so there more flat. You could add a little holt glue under them to get better contact.