You are doing something wrong, that is not knowing how to properly run and LED. Read an LED tutorial so you know what you need to do, hooking an LED to straight power source may work in some cases, however if you want to protect your LEDs you will need a RESISTOR. There are plenty of LED calculators on google searches, so look em up.
Not really trying to be a jerk here, it is just extremely dangerous to play with electricity without an appreciation for the unexpected. Granted it is hard to hurt yourself with a AA battery, it is easy to hurt yourself with a 200W 360 PSU, not to mention risk of fire.
In order to light those LEDs up it will take 2AA bateries in series for the red and yellow LEDs, and probly 3AA in series in order to light up the blue and Green LEDs. In the case of AA batteries you can probly just hook them straight up without a resistor if you are just briefly testing, because the internal resistance of the batteries is high enough, it wont allow for them to draw an excessive amount of current. However you will still be drawing too much current for the LEDs to last for their whole expected lifetime.
Please in the future research what you are doing, it is more of a safety issue than anything else