I'm running Murph's bar adapters for tubular bars, and have on both old Conni's, there is no reason to mess with a clearance hole for the dowel pin in the switch assembly ever, when I installed, I installed the switch assembly loosely, and placed a small socket on the back of the housing, over the dowel pin, and tapped it gently with a hammer, displacing the dowel pin out from the switch housing till it fit the bars. the pin remains tightly held in the housing, albeit displaced, and can be returned to use at a later date if so desired.
To insure the switch housing does not scare you, simply place 2 wraps of plastic electrical tap around the bar in the area the switch housing clamps onto, it will never revolve when tightened. Been running mine like that for years. Even with an oversized hole, the tape will offer wayyyy more than enough grip to prevent movement.