Please give us details on which Xbox you are talking about.
There is the first gen Xbox, usually black plastic blocky with a green "X" emblem on the top.
Then there are the 360 models.
There have been a few revisions, lets just call them phat and slim.
phat has an external harddrive fastened to one end, and the slim has an internal harddrive.
Heat problems are almost non-existent with the original XBox consoles.
Heat problems were rampant on the first generation of the phat Xbox 360 consoles.
Heat problems were mitigated on the following generations of the phat Xbox 360 consoles,
Heat problems have been mitigated additionally on the slim Xbox 360 consoles.
The heatsink can not really be improved on based on the very very tight spaces involved within the console. This is true for all Xbox consoles.
The heating problems present in the 360 consoles usually result in the GPU's connection to the board being stress fractured resulting in the 'Red Ring of Death'. (Error Code.)
This is commonly resolved with physically clamping the GPU (and CPU) to the board much harder to physically hold them together. This is not a 100% fix, but has lots of success.
Other users use external case modifications to force more air through the console to help keep the temperature down. This is preventative, and will not 'repair' a broken console.
I have personally opened my 360, replaced the fans with some that provide additional airflow at the expense of being louder, and inserted a small divider to divide the main air channel between the CPU and GPU. (As the same fan assembly covers both) My box still died from the stress fractures mentioned above.
Heat is caused by the processor during operation, as it's a collection of millions of transistors. They do not switch instantly, so everytime they switch they generate a little bit of heat. When you do lots of work on a processor, it switches more often generating more heat. If you overclock the processor it will switch even faster... you guessed it. making more heat. Additionally, the voltage to the processor is sometimes increased to compensate for the losses that are caused by switching much faster than originally designed... and yes, you guessed it. Higher voltage equals more heat.
So to combat this, you have two options:
-Make less heat.
-Remove the heat faster than it can build up.
Make less heat
This is common among cell-phones and other mobile devices. The goal here is to slow down the processor and run it at it's slowest possible setting that can still accomplish your tasks. The processor will automatically speed itself up when your tasks begin to require more processing power. So these types of applications are not always 'slower'. They just don't sprint if they can do the same job walking.
I personally limit my phones processor to just under 2/3 of it's maximum to prevent games and other apps from making it run too hot. This has had great results, but there is a trade-off.
A slow processor takes longer to do a complicated task then a fast processor. A slow processor will use less power... but for a longer time... This does not help too much with 'battery life' because getting the job done in a fraction of the time and having the processor sleep will use a similar amount of power than having a weak processor to the same job over a few seconds before going to sleep.
Remove heat
This is common on pretty much anything that you can possibly fit a heatsink to. Laptops included. (Although the heatsink is smaller)
The limitation here is that the heatsinks are bulky. The simple addition of a heatsink will add more surface area to the processor which will help the heat to disperse at a faster rate. The surface area is important, it's the reason for all of the fins on a heatsink.
You can make a heatsink more efficient by improving the connection it has to the processor. This is done with heatsink compound/paste... or as mentioned above, toothpaste
This fills any small air gaps in between the processor and heatsink which will help to carry the heat away from the processor. Different compounds are better than others... but even with a perfect compound, you will still be limited by the heatsink's ability to disperse the heat. Remember the paste will help it to work to it's potential but will not make it work better than it was meant to.
Then you can help it disperse the heat better by improving air-flow. Heatsinks simply allow the air to pick up the heat and carry it away... so if you allow for more airflow you can cool the processor faster.
You can also use forced air cooling from a fan to force much more air across the heatsink to dissipate even more heat into the air. Remember though, that the cooler the air is, the more heat it can take. If you have a fan inside a cabinet, it may be able to move the air, but the air will warm up and get recirculated as warm air which will not carry away the heat as well. It is NOT a good idea to operate devices which generate heat inside a cabinet. You can also go to extremes and use liquid cooling. Many liquids can take a lot more heat, a lot more efficiently than air. So this will allow the heatsink to operate even better than with air... you just need a way to keep any harmful liquids away from the electronics. Of course, when using liquid, you can now use a smaller heatsink.
Then you can go to extremes and
submerge everything in liquid, or use things like liquid nitrogen or small Air Conditioning coils...
