My guess is that a latching switch has suddenly become a momentary switch.
I say the opposite: The power button for a PC is usually a momentary switch. press it shortly and the PC will turn on. Keep it pressed and the PC will turn off.
Supported by this observation:
It still turns off even if I keep the pressure.
I think the wiring is wrong. I see 2 wires only and 2 contacts of the switch connected:

To the best of my knowledge the power button on a PC works by connecting the power-on input on the motherboard to ground. The LED is powered by an extra 3rd wire. Here it seems the LED is connected across the button's contacts and thus pulls the power-on pin on the motherboard low enough to be detected as logic low, thus turning the PC off.
Use only one of the two pins for the red wire. I don't know for sure which one, my guess is the one opposing the black wire. If that works, you'll be left without indicator LED. If it doesn't work, find out which pins are for the switch and which pin is for the LED.
Use then a 3rd wire to connect the LED. consult your motherboard documentation where to connect this 3rd wire. It is likely the 3rd unused pin in the connector.
