Well, I was watching the three-part finale in Season 3 - the one with the Doctor/Master story arc - and in the Sound of Drums, I think, the Master [as Prime Minister Saxon] is getting down to the business of "government" and he throws all the folders up in the air - well, the Master is psychotic - so they fall down everywhere and the others all stare at him and he says, "Well, go on, crack a smile," and "It was funny, wasn't it?" and then he illustrates the point with his 'funny' vs. 'not funny' faces... the "sometimes people take him a bit too seriously" part just reminded me of that.