Use async-mutex to avoid start/stop being racy

With both being async it's hard to follow what would happen if both run at the same
time, so just prevent it and serialize start/stopping instead.