Actually it's the other way around. If you connect it to the speakers, the LEDs will always be on. The point of the amp is to make the LEDs come on only with sound.
I've had a similar problem before. Do not put the resistor on the output of the amp, but instead put the two required resistors on the audio input lines. That May help.
This may not be the problem, but I've accidentally hooked the LEDs onto the amp backwards and it results in what your describing. You could try switching the direction of the LEDs and see if that helps.