First, I am reaching with this one, so nothing is for certain.
USB ports have one line that is a 5 VDC signal. This is what powers the devices connected to it. A mouse has a lot less current draw than does a keyboard. That was why I wanted you to unhook everything from the USB ports and try just the keyboard. Less load on the 5 VDC part of the power supply.
While you are in the computer removing the card, try unhooking the power connections to all of the other devices except for the harddrive, and any power lead that goes to the motherboard.
Then try starting the computer up again.
Let me know how it goes.