Looking up means you are thinking about a picture, looking down means you think about what someone said. It's possible that someone does the opposite but you will notice it depending on your questions in a conversation. When I ask: What did she say? Most people will look down and think about it. When I ask: How was the vacation? Most people will look up because they think about what they saw.
You can read these things when you study NLP. (Neuro Linguistic Programming)
!BBH !HEARTBEAT !ALIVE