What U4 did you replace? The U4 on the top board is just a switch, then U4 on the bottom board is for driving the left side Rumble motors.
Sticks stuck down/right would be no ground on the POTs or the lines shorted to voltage, but then they wouldn't move to the left at all. I'd reflow U1 on the bottom board, that thing has been known to cause weird issues.