Autonomous systems especially those driven under optimal control are usually associated with objective functions to describe their goals/tasks in specific missions. Since they are usually unknown in practice especially for complicated missions, learning such objective functions is significant to autonomous systems especially in their imitation learning and teaming with human. In this talk we will introduce our recent progress in objective learning based on inverse optimal control and inverse optimization, especially their applications in human motion segmentation, learning from sparse demonstrations, and learning with directional corrections. We will also present an end-to-end learning framework based on Pontryagin Principle, feedbacks and optimal control, which is able to treat solving inverse optimization, system identification, and some control/planning tasks as its special modes.