Contact Information
Email: icleen@gmail.com iain.lee@utah.com
Phone: (530) 867-7768
Email: icleen@gmail.com iain.lee@utah.com
Phone: (530) 867-7768
Instead of learning a policy from scratch to perform robotics tasks, we propose leveraging known motion planning algorithms in conjunction with reinforcement learning to overcome unpredictable state and action dynamics. Building off of the work of Residual Policy Learning or Residual Reinforcement Learning for Robot Control we proposed to have a base policy take actions in the environment with each action modified slightly by a learned residual policy. This would allow the policy to learn the important parts of the state space more efficiently and then learn to modify the base policy so as to improve its efficacy towards the final goal. Our target environment was pick and placement of objects using a 6dof robotic arm. Residual Learning was never proven as a viable method for the task, however. Recent studies such as MaxDiffRL propose that learned policies struggle when the action control matrix is modified so that exploring some parts of the state space are more difficult than others. This could explain Residual RL's lack of performance on the pick and place task. Current work is on attempting to verify this hypothesis and using MaxDiffRL to solve the residual learning paradigm.
Robots often struggle with precise pick and place tasks. Building a Lincoln Log cabin is a particularly difficult task as it requires precise estimations of log locations on the work surface, as well as precise placement in the target cabin location. This is additionally made difficult when the robot hand adjusts the pose of the log when picking, making the offset between hand and log an unkown variable. This project was to see how far we could get in this task utilizing only off the shelf algorithms. As you can see in the video, we were occasionally able to place logs correctly. Increased success rate would require a better estimation of the log post-grasp and is left as a future project.
I helped develop in-house software to perform trajectory optimization under a variety of optimization algorithms, including gradient descent, gaus-newton's method, and BFGS. We developed these with both the Augmented Lagrangian and Penalty methods for optimizing over constraints, the constraints in our case being object collisions. See ll4ma-opt-sandbox for details.
My most recent education was at the Univeristy of Utah studying AI and robotics. While I was initially in the PhD program I eventually decided to switch to a masters. I studied under Tucker Hermans in a variety of projects listed above. My interests lie primarily in machine learning methods (including reinforcement learning), optimization, decision making, and pattern recognition. During the program I TA'd for the computer vision and data science courses, both topics I am very comfortable with.
I discovered that I enjoyed the problem solving involved with computer science during my first year at BYU. I quickly decided that this was the major I wanted. I then did a variety of undergraduate research projects under multiple professors. The most memorable were developing Japanese Kanji recognition using deep learning models and historical document segmentation using line segmentation algorithms. The latter took place during a project in which I helped supervise other undergraduate students in developing an app for automatic scanning of US census records from the early 1900s. The intent was to do data collection from these census records. I graduated before the project was complete.
I love the outdoors and particularly rock climbing. I love the puzzle solving required, as well as the opportunity to enjoy nature in a unique way!
I love the wind on my face and the opportunity to see new scenery. Biking through national parks like Grand Teton is particularly fun!
My hot wife finally convinced me to start skiing and I've come to enjoy it, even if I'm not very good.