Should all flashings be replaced when I have my roof replaced?

Yes, the time to address all the roof issues was when the roofer was replacing the roof.  Normally all flashings should be inspected and if any are damaged or rusted they should be replaced when the roof is replaced.  The cost would be much less at that time, not to mention possible water damage and cost to have it repaired at a later date.