21V is only 17% more than 18V. That's not bad for a small transformer.
In the days when power supplies (wall warts, plug packs) consisted of a simple transformer, rectifier and filter, it was totally normal for the output voltage to be significantly higher than what was marked until a significant load was placed on it. (I've seen almost double!)
These days most such devices are switchmode regulated and supply an almost exact voltage over the entire load range.
The transformer is not rated to give you two 18V rails. It will be described as having two 18V windings. If you were able to get a
datasheet on the transformer you might find that it would describe the change in voltage with load.
If your preamp has a regulator, that will take care of differing voltages under various loads. If it does not have a regulator (and that is not uncommon -- especially in power amplifiers) then the circuit will be designed to handle the varying supply voltage. 18VAC will give A DC rail of about 25.5V. With your off-load voltage reading of 21V, that will probably rise to about 30V unloaded.