Adaptive Dynamic Programming (ADP) has received increasing attention recently. ADP scheme is a design that approximates dynamic programming in the general case, i.e., approximates optimal control over time in noisy, nonlinear environments. There are many engineering problems in practice which can be formulated as cost maximization or minimization problems. Dynamic programming is a very useful tool in solving these problems. However, it is often computationally untenable to run dynamic programming due to the backward numerical process required for its solutions. Over the years, progress has been made to provide approximate solutions to dynamic programming. The idea is to approximate dynamic programming solutions by using neural networks to approximate the cost function. The methodology is a very useful tool for building intelligent agents/controllers in almost any environment. This talk will review the theorectical development of ADP. Details about the training of the neural networks used in the present design will also be presented. The pole balancing (inverted pendulum) problem will be used as the benchmark in this presentation to show the applicability of ADP.

Derong Liu received his Ph.D. from University of Notre Dame in 1994, he has been with the University of Illinois at Chicago since 1999 as an Assistant Prof., Associate Prof., and Full Prof. Prof. Liu is an IEEE Fellow. He received CAREER award from Nature Science Foundation in 1999. Ha has supervised 9 Ph.D students and 6 M.S. students. He served as AdCom Member of IEEE Computational Intelligence Society, Letters Editor of IEEE Trans. On Neural Networks, Asso. Editor of IEEE Trans. On Signal Processing, Asso. Editor, of IEEE Trans. On Circuits and Systems: I: Fundamental Theory and Applications, Editor of Automatica, etc. He was General Chairs of two IEEE Conferences and four Program Chairs of Int. Conferences. He has published 5 books, 6 book chapters, 57 jounal papers, and 100 conference papers up to now.