Ants in a wormery will eat worm eggs and therefore should be removed.
You will need to disturb the nest and, if possible, remove the queen ant (a pair of gloves are advisable for this step!). The queen ant is distinguishable by the fact that she is much larger than the other ants.
A light sprinkling of cinnamon will help to disperse the ants, but won't harm your worms.
Once the nest is removed, place the wormery legs in pots of water and smear a strip of Vaseline around each leg. It is a good idea to adopt these defensive steps in early spring before the ants have a chance to get into your wormery