Someone previously told me that Runescape didn't work under Firefox.
In any case, the difference in behavior may be attributed to Firefox using Sun Java and IE using MS Java Virtual Machine. However, it could also be due to some ActiveX control as Firefox wouldn't support that either. Then again, it could simply be that Firefox is missing a plugin that IE's already got.
Can't say for certain though since I've never played the game.