Fix rare races when using workers.