Do not move!
Seriously, you simply need more time in the water, this will work itself out. I am not a physiologist so I will let someone else fill in on which parts of our body it is that needs tuning - but not falling off the board while sitting is like not falling of the bike when cycling at stroll speed. Not easy in the beginning, but after a while it is something you do not even think about, you just do it. It is an equilibrium that somehow bypasses the brain and disappears into the autonomous systems in your body or something like that.
(Looking at how the others are doing it is a good idea).
PS> if you fall off, remember it always looks kind of silly, so just laugh of your mistakes and go on to the next one without letting 'em defeat you