You pretty much got it, you can do it without removing the shield though. Raise the shield all the way up and use a small Allen wrench to press the lil buttons in on the rivets and you can pull them right out.
You don't have to remove the instrument trim panel either.