To Downgrade a Die you simply replace the indicated Die in the Pool with the next Die as per the following progression system:
If you have been granted a Downgrade but all of the Die in the Pool are already D4’s, then nothing happens. A Die cannot be Downgraded below D4.