The phrase is usually a humorous way of saying that one was not sitting idly, but engaged constructively in contemplation or letting one's mind wander.
We demystify authorship controversies, assassination conspiracies, theories of extra-terrestrial shindigs, even painful social demands, by letting their proponents speak out, not by censoring them.