I think "pillar drill" is actually Swedish shining through. I think (but may be wrong) they're drill presses in the UK as well. I've seen both the hand-held-drill and Dremel variants, but a hand-held usually has a bit more oomph than a Dremel. Main reason I was thinking about them is that they (in my experience, at least) tend to both allow you to use less force as well as tending to improve the precision. Although the non-dedicated units are somewhat wobbly, but should still allow you to get ~1 mm accuracy.

Plastic, pretty much any plastic (except possibly fibre-glass reinforced ABS), I would worry about stress cracks. And you REALLY don't want the reinforced ABS, as it eats about one tool-steel drill every, oh, 100 mm of material drilled through (yep, spent a summer machining the stuff, it's lovely but really hard on tools).

If I decide to make another arcade controller for home (I think the old one is dead by now), I'll probably buy one of the "takes a hand-held" drill press jobbies. It worked fine doing it by hand, but that's enough holes and boring (ahaha) enough that I'd rather spare my back from the hunching. On the other hand, that's somewhere in the region of 25 holes. On the gripping hand, it was in wood (Ivar shelving, so it hooked on nicely just above my Ivar computer shelf thing).