Dependency-aware action planning for smart home

by Jongjin Kim, Jaeri Lee, Jeongin Yun, U. Kang

How can a smart home system control a connected device to be in a desired state? Recent developments in the Internet of Things (IoT) technology enable people to control various devices with the smart home system rather than physical contact. Furthermore, smart home systems cooperate with voice assistants such as Bixby or Alexa allowing users to control their devices through voice. In this process, a user’s query clarifies the target state of the device rather than the actions to perform. Thus, the smart home system needs to plan a sequence of actions to fulfill the user’s needs. However, it is challenging to perform action planning because it needs to handle a large-scale state transition graph of a real-world device, and the complex dependence relationships between capabilities. In this work, we propose SmartAid (Smart Home Action Planning in awareness of Dependency), an action planning method for smart home systems. To represent the state transition graph, SmartAid learns models that represent the prerequisite conditions and operations of actions. Then, SmartAid generates an action plan considering the dependencies between capabilities and actions. Extensive experiments demonstrate that SmartAid successfully represents a real-world device based on a state transition log and generates an accurate action sequence for a given query.