Lee Clement

About Me

Hi! I'm a final-year Ph.D. Candidate in the Space and Terrestrial Autonomous Robotic Systems Lab (STARS) at the University of Toronto, where I'm advised by Prof. Jonathan Kelly. I'm also a Postgraduate Affiliate of the Vector Institute in Toronto. My research focuses on the intersection of machine learning and robotics with an emphasis on deep learning for visual localization. More broadly, I'm interested in the ethics of data and automation, and the intersection of technology and society, particularly approaches that center the impacts of technology on marginalized communities. I have lectured on topics in robotics, state estimation, computer vision and machine learning at the undergraduate and graduate levels, and co-created the State Estimation and Localization course of the Coursera Self-Driving Car specialization.

I'm originally from Winnipeg, Manitoba (affectionately: Winterpeg, Manisnowba). In my spare time I can often be found soaking up Toronto's vibrant music scene, perfecting my sourdough, or binge-listening to podcasts.

Download CV Contact Me

Research

Life-long Autonomy

The world around us is constantly changing. Robots need tools for coping with environmental change if they are to operate autonomously over long periods of time without human intervention.

Deep Learning as a Sensor

Deep learning is a powerful tool, but only one of many in the roboticist's toolbox. By thinking about deep learning as a kind of extra sensor, we can combine it with existing tools to make robots work better.

Sensor Fusion & Estimation

Real-world robots rely on a suite of sensors to understand the world around them. To fuse all of this data into a consistent belief, we need careful sensor calibration and probabilistic models.

Publications

Journal & Magazine Articles

Where Do We Go From Here? Debates on the Future of Robotics Research at ICRA 2019
Lee Clement, Valentin Peretroukhin, Matthew Giamou, John Leonard, Hadas Kress-Gazit, Jonathan How, Michael Milford, Oliver Brock, Ryan Gariepy, Angela P. Schoellig, Nicholas Roy, Hallie Siegel, Ludovic Righetti, Aude Billard and Jonathan Kelly
IEEE Robotics and Automation Magazine (RA-M), 26(3): 7-10, 2019

How to Train a CAT: Learning Canonical Appearance Transformations for Direct Visual Localization Under Illumination Change
Lee Clement and Jonathan Kelly
IEEE Robotics and Automation Letters (RA-L), 3(3): 2424-2431, 2018
Presented at the IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia, 21 - 25 May 2018
 Preprint   Code   Poster   Video 

Inferring Sun Direction to Improve Visual Odometry: A Deep Learning Approach
Valentin Peretroukhin*, Lee Clement* and Jonathan Kelly
International Journal of Robotics Research (IJRR), 37(9): 996–1016, 2018
*Equal contribution
 Code 

Robust Monocular Visual Teach and Repeat Aided by Local Ground Planarity and Colour-Constant Imagery
Lee Clement, Jonathan Kelly and Timothy D Barfoot
Journal of Field Robotics (JFR), 34(1): 74–97, 2017
 Preprint 

Conference Articles

Matchable Image Transformations for Long-term Metric Visual Localization
Lee Clement, Mona Gridseth, Justin Tomasi and Jonathan Kelly
Presented at the Workshop on Image Matching, IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, USA, 13-19 June 2019
 Preprint   Poster 

Reducing Drift in Visual Odometry by Inferring Sun Direction using a Bayesian Convolutional Neural Network
Valentin Peretroukhin*, Lee Clement* and Jonathan Kelly
In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Singapore, 29 May - 3 June 2017
*Equal contribution
 Preprint   Code   Slides   Poster   Video 

Improving the Accuracy of Stereo Visual Odometry Using Visual Illumination Estimation
Lee Clement, Valentin Peretroukhin, and Jonathan Kelly
In Proceedings of the 2016 International Symposium on Experimental Robotics (ISER), Tokyo, Japan, 3 - 6 October 2016
Toyota Student Participation Award; Invited to IJRR Special Issue
 Preprint   Slides 

Entropy-based Sim(3) Calibration of 2D Lidars to Egomotion Sensors
Jacob Lambert, Lee Clement, Matthew Giamou and Jonathan Kelly
In Proceedings of the 2016 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Baden Baden, Germany, 19 - 21 September 2016
Best Student Paper Award
 Preprint   Slides 

PROBE: Predictive Robust Estimation for Visual-Inertial Navigation
Valentin Peretroukhin, Lee Clement, Matthew Giamou and Jonathan Kelly
In Proceedings of the 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany, 28 September - 2 October 2015
 Preprint   Slides   Video 

The Battle for Filter Supremacy: A Comparative Study of the Multi-State Constraint Kalman Filter and the Sliding Window Filter
Lee Clement*, Valentin Peretroukhin*, Jacob Lambert and Jonathan Kelly
In Proceedings of the 12th Conference on Computer and Robot Vision (CRV), Halifax, Nova Scotia, 3 - 5 June 2015
*Equal contribution
 Preprint   Code   Slides 

Get to the Point: Active Covariance Scaling for Feature Tracking Through Motion Blur
Valentin Peretroukhin, Lee Clement and Jonathan Kelly
Presented at the Workshop on Scaling Up Active Perception, IEEE International Conference on Robotics and Automation (ICRA), Seattle, Washington, 26 - 30 May 2015
 Preprint   Slides   Poster 

Monocular Visual Teach and Repeat Aided by Local Ground Planarity
Lee Clement, Jonathan Kelly and Timothy D Barfoot
In Proceedings of the 10th Conference on Field and Service Robotics (FSR), Toronto, Ontario, 24 - 26 June 2015
 Preprint   Slides   Video 

Monocular Vision for Long-range Visual Teach and Repeat in Unstructured Environments
Lee Clement, Jonathan Kelly and Timothy D Barfoot
Presented at the NSERC Canadian Field Robotics Network (NCFRN) and Conference on Computer and Robot Vision (CRV) Joint Poster Session Montreal, Quebec, 10 May 2014.
 Poster 

Implementation of a Nanosatellite Attitude Determination and Control System for the T-Sat1 Mission
Brady Russell, Lee Clement, Joshua Hernandez, Ahmad Byagowi, Dario Schor and Witold Kinsner
In Proceedings of the Canadian Conference on Electrical and Computer Engineering (CCECE). Regina, Saskatchewan, 5 - 8 May 2013

Teaching & Service

Teaching

Course Instructor – ROB 501 (Computer Vision for Robotics) – Fall 2018
This course provides an introduction to aspects of computer vision specifically relevant to robotics applications (i.e., robotic vision). Topics include the geometry of image formation, basic image processing operations, camera models and calibration methods, image feature detection and matching, stereo vision, structure from motion, and 3D reconstruction from a moving platform.

Course Instructor – AER 521 (Mobile Robotics and Perception) – Winter 2018
This course addresses fundamentals of mobile robotics and sensor-based perception for applications such as space exploration, search and rescue, mining, self-driving cars, unmanned aerial vehicles, autonomous underwater vehicles, etc. Topics include sensors and their principles, state estimation, computer vision, control architectures, localization, mapping, planning, path tracking, and software frameworks.
Previously: Teaching Assistant – Winter 2016-17

Subject Matter Expert – Coursera (Self-Driving Cars Specialization) – 2018
This Specialization provides a comprehensive understanding of state-of-the-art engineering practices used in the self-driving car industry. Learners interact with real data sets from an autonomous vehicle through hands-on projects using the open source simulator CARLA. As a Subject Matter Expert, I helped create and script video lectures on topics in State Estimation and Localization for Self-Driving Cars.

Teaching Assistant – ROB 301 (Introduction to Robotics) – Fall 2015-18
This course is intended to provide an introduction and a very interdisciplinary experience to robotics. The structure of the course is modular and reflects the perception-control-action paradigm of robotics. Applications addressed include robotics in space, autonomous terrestrial exploration, biomedical applications such as surgery and assistive robots, and personal robotics. The course culminates in a hardware project centered on robot integration.

Professional Activities & Community Service

Organizer – Debates on the Future of Robotics Research – ICRA Workshop, 2019
The Debates on the Future of Robotics Research Workshop brings together prominent researchers and industry leaders to formally debate key issues affecting robotics as an academic discipline and its broader social and economic contexts.

Postgraduate Affiliate – Vector Institute – 2019-20
The Vector Institute drives excellence and leadership in Canada’s knowledge, creation, and use of artificial intelligence (AI) to foster economic growth and improve the lives of Canadians.

Invited Participant – RSS Pioneers Workshop – 2018
RSS Pioneers is a day-long invitation-only workshop for senior graduate students and postdocs, held in conjunction with Robotics: Science and Systems, that seeks to bring together a cohort of the world’s top early career researchers in all areas of robotics.

Student Mentor – Inclusion@RSS – 2018
Inclusion@RSS focuses on programs that increase and sustain a broader participation in the robotics research community of groups traditionally underrepresented in robotics (including but not limited to women, LGBTQ+, underrepresented minorities, and people with disabilities), especially people early in their studies and career.

Graduate Student Representative – Engineering Faculty Council – 2017-18
Faculty Council sets and approves academic policy, principles, priorities, and the general direction for the teaching and research activities of the Faculty of Applied Science and Engineering.

Graduate Student Representative – Community Affairs & Gender Issues Standing Committee – 2017-18
The Community Affairs & Gender Issues Standing Committee in the Faculty of Applied Science and Engineering seeks to improve and to recommend on strategies related to student recruitment and outreach and quality of life within the Faculty community, including such matters as the student experience, gender issues, diversity, safety and security, and personal conduct.

President – Aerospace Students' Association – 2016-17
The Aerospace Students' Association represents graduate students at UTIAS and organizes athletic, social, academic and professional events. As President, I coordinated a team of 14 students to organize a variety of fun events like Trivia Night, Pancake Breakfasts, BBQs, and Camping Trips, and collaborated with the UTIAS administration and other graduate student associations to enrich the overall experience of graduate students at UTIAS.
Previously: Social Coordinator – 2015-16

Co-founder / Aerospace Representative – Graduate Engineering Council of Students – 2016-17
GECoS acts as a forum for representatives from all Engineering Graduate Student Associations at UofT to collaborate on academic, social and professional events, discuss issues that broadly affect Engineering graduate students, and represent the general interests of Engineering graduate students to the Faculty and University.

Student Member – UTIAS Student Experience Committee – 2016-17
The SEC is responsible for gathering data about the UTIAS student body’s experiences at the Institute and making a report to the Director summarizing the data and providing suggestions for improvement.

Re-founder / Director – SEDS-Canada – 2014-17
SEDS-Canada (Students for the Exploration and Development of Space) is a national student-led not-for-profit corporation dedicated to improving the Canadian space sector. As a member of the Board of Directors, I helped direct and oversee the organization’s activities, which included multiple student competitions and an annual conference.