Reinforcement learning for POMDP using state classification

Le Tien Dung, Takashi Komeda, Motoki Takagi

Research output: Contribution to journalArticlepeer-review

16 Citations (Scopus)


Reinforcement learning (RL) has been widely used to solve problems with a little feedback from environment. Q learning can solve Markov decision processes (MDPs) quite well. For partially observable Markov decision processes (POMDPs), a recurrent neural network (RNN) can be used to approximate Q values. However, learning time for these problems is typically very long. We present a new combination of RL and RNN to find a good policy for POMDPs in a shorter learning time. This method contains two phases: firstly, state space is divided into two groups (fully observable state group and hidden state group); secondly, a Q value table is used to store values of fully observable states and an RNN is used to approximate values for hidden states. Results of experiments in two grid world problems show that the proposed method enables an agent to acquire a policy with better learning performance compared to the method using only a RNN.

Original languageEnglish
Pages (from-to)761-779
Number of pages19
JournalApplied Artificial Intelligence
Issue number7-8
Publication statusPublished - 2008 Aug
Externally publishedYes

ASJC Scopus subject areas

  • Artificial Intelligence


Dive into the research topics of 'Reinforcement learning for POMDP using state classification'. Together they form a unique fingerprint.

Cite this