Understanding how tasks progress over time enables digital assistants to help with current activities and support future activities. Imbuing assistants with the ability to track task progress requires machine-learned models. In this paper, we describe an ongoing effort to collect signals from Cyber, Physical, and Social (CPS) activities, together with human assessments of task progression, to serve as a benchmark for training and testing such models. Collecting this data over time is inherently challenging in the daily sensing scenario. Consequently, lessons learned from our ongoing data collection are discussed to provide more insights for future research innovations in task intelligence.