My best guess on how add/remove multiple supports currently works is simplify clicks every time a frame is rendered while the mouse is held down. This is fine for small parts or when using large support pillars, but when slicing models for a printer with a bed area of 762x762mm fine control of supports becomes a time consuming task. A better implementation would be to delete all support pillars that the mouse moves over regardless of frame rate.
Can you post a video showing the behavior you are seeing? The frame rate of your computer shouldn't affect the support removal at all.
If your model is really large, it's possible when you zoom very far out the supports are just too small to even see. If that's the case, maybe using a slightly larger pillar width would help?