You will hate it. (See pics.)
It is one of the BEST trips you can do! I haven't done between Cache Creek and Lillooet but have done Whistler to Lillooet many times and Kamloops to Cache Creek as well. The Duffy Lake road has been repaved - completed last year. What a difference! It is slow with a lot of twisties and switchbacks. 2+ hours for the Whistler to Lillooet part. It would make for a looong day to do it round trip unless you didn't want to stop and see anything. No gas between Lillooet and Pemberton. Otherwise you are fine.
Do it! Enjoy! But wait for a nice day or you won't see squat!