Educational Virtual Worlds (EVWs) provide an interesting and engaging educational platform to promote student learning. Virtual classrooms should be designed to be responsive and adaptive enough to judge the emotional level of the learner and should be capable of providing required support and feed-back to maintain the engagement of the learner to achieve their learning goals. Virtual agents with affective capabilities have potential to improve the effectiveness and efficiency of the e-learning and virtual learning platforms. This research study will explore understanding of learners’ emotional states that evolve during the learning process within a EVW by drawing on theories related to epistemic emotions and develop approaches to recognise and intervene in a timely manner based on the individual’s feelings being experienced at that moment. To derive that emotional state, this paper also presents a multi-data and multimodal approach to judge the underlying emotional state of the learner within EVW and the most appropriate intervention for that learner.