Simply put, you can't. When I attempt to change the color/category of an occurrence, I get a pop-up message advising, "You can only change categories by opening the series."
This leaves you with two choices:
(1) Establish one color/category for the series and leave it. OR
(2) Change the color/category prior to each event, but this will involve sending out updates to each participant, which can grow wearisome, especially to those participants who already receive too much email.