research

home contact publications cv
koan simra.net

I'm currently an applied science manager at MSR AI. I'm currently exploring problems in privacy-preserving machine learning, conversational task assistance, and instrumenting group interactions (meetings) towards building a better understanding of the relationships between interpersonal behaviors and meeting effectiveness.

From about 2008 to 2017 I was an applied researcher and occasional research manager in Microsoft's Safety Platform, where we did a lot of interesting work on protecting users online- spam prevention, family safety, account protection, fraud and scam prevention and phishing and malware detection.

Message from a past life (circa 2005): My research explores problems in mobile robot localization and environment learning using vision and range sensors. In short, I'm looking to answer questions like: Given a map, and a sensor reading, what is the most plausible place I might be in? How can I move to improve my confidence about where I am? What are the best places to move to to build a map? What is the best representation for a map? How can attention be used to simplify the map-building process?

Previously I was a Senior Robotic Vision Scientist at Braintech, Inc.

For a while in 2006 I was a vision researcher at Microsoft adCenter Labs.

From Jan 2004-July 2006 I was at UBC, working as a postdoc with Jim Little and David Lowe.

In January, 2004, I defended my Ph.D. Thesis, entitled On Visual Maps and their Automatic Construction.

In the fall of 2003, I worked at UofT with Sven Dickinson.

In 2000, I submitted my Ph.D. Proposal on Bayesian Exploration under the supervision of Prof. Greg Dudek at the Centre for Intelligent Machines, McGill University.

My Master's thesis is entitled "Mobile Robot Localization from Learned Landmarks". You can browse my thesis online here.

Check out the mobile robotics lab (my former lab) for more info.

publications

journal publications patents theses and proposals conference and workshop publications invited publications abstracts technical reports

My (out-of-date) BibTeX file is here.

refereed journal publications

Multi-Device Digital Assistance, Ryen W. White, Adam Fourney, Allen Herring, Paul Bennett, Niru Chandrasekaran, Robert Sim, Elnaz Nouri, Mark Encarnacion, and Nalin Singal. Communications of the ACM (CACM), Vol: 62, Issue: 10, Pages: 28-31. [pdf]

Autonomous vision-based robotic exploration and mapping using hybrid maps and particle filters, R. Sim and James J. Little, Image and Vision Computing, Volume 27 Issue 1-2, January, 2009 Pages 167-177. ACM Link

A study of the Rao-Blackwellised particle filter for efficient and accurate vision-based SLAM, R. Sim, P. Elinas and J. J. Little, International Journal of Computer Vision/International Journal of Robotics Research Special Joint Issue on Vision in Robotics,vol 74, no 3, pp 303-318, Kluwer, Sept 2007

Landmark Selection for Vision-Based Navigation, P. L. Sala, R. Sim, A. Shokoufandeh and S. J. Dickinson. IEEE Transactions on Robotics, vol 22, no 2, pp 334-349. IEEE Press. April 2006.[preprint]

Learning generative models of scene features, R. Sim and G. Dudek, International Journal of Computer Vision vol 60, no 1, pp 45-61, Kluwer, 2004. Pre-press version: [pdf] [Kluwer on-line ]

Learning environmental features for pose estimation, R. Sim and G. Dudek, Image and Vision Computing, vol 19, num 11, pp 733-739, Elsevier Press, 2001. [Reprints available on request] [ postscript (650kb)]

patents

Brokered Authentication with Risk Sharing, Robert A Sim, Akash A Shah, Jisheng Liang
US10652282

Empathetic personal virtual digital assistant, Nirupama Chandrasekaran, Robert A Sim, Ryen W. White, Nikrouz Ghotbi
US10522143

Risk assessment modeling, Luke Abrams, David J. Steeves, Robert Alexander Sim, Pui-Yin Winfred Wong, Harry Simon Katz, Aaron Small, Dana Scott Kaufman, Adrian Kreuziger, Mark A. Nikiel, Laurentiu Bogdan Cristofor, Alexa Lynn Keizur, Collin Tibbetts, Charles Hayden
US9779236B2

Chunk-Based File Acquisition And File Reputation Evaluation, Robert Alexander Sim, Christian Seifert,Anthony Penta, Elliott Jeb Haber, Tomasz Kasperkiewicz
US9398036

Generating stimuli for use in soliciting grounded linguistic information, William B. Dolan, Christopher I. Charla, Christopher B. Quirk, Christopher J. Brockett, Noelle M. Sophy, Nicole Beaudry, Vikram Reddy Dendi, Pallavi Choudhury, Scott T. Laufer, Robert A. Sim, Thomas E. Woolsey, David Molnar
US9116880B2

Using the ability to speak as a human interactive proof, Chad Mills, Robert Sim, Scott Laufer, Sung Chung
US9390245B2

Intelligent Information Display, Li Li, Robert A. Sim, Tarek Najm, Ying Li, Yi Chen.
US20080262909A1

theses and proposals

On Visual Maps and their Automatic Construction, Robert Sim
Ph.D. Thesis, 170 pages. [ pdf (7.0Mb)]

Mobile robot localization from learned landmarks, Robert Sim
Master's Thesis, McGill University, 66 pages. July, 1998 [ postscript (2.5 MB)]

To Boldly Go: Bayesian Exploration for Mobile Robots, Robert Sim
Ph.D. Proposal, 24 pages. [ postscript (278kb)]

refereed conference and workshop publications

Privacy Regularization: Joint Privacy-Utility Optimization in Language Models, Fatemehsadat Mireshghallah, Huseyin A. Inan, Marcello Hasegawa, Victor Rühle, Taylor Berg-Kirkpatrick, Robert Sim
2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2021). pdf, arxiv

MeetingCoach: An Intelligent Dashboard for Supporting Effective and Inclusive Meetings, Samiha Samrose, Daniel McDuff, Robert Sim, Jina Suh, Kael Rowan, Javier Hernandez, Sean Rintel, Kevin Moynihan, Mary Czerwinski
CHI 2021. pdf

Leveraging Structured Metadata for Improving Question Answering on the Web, Xinya Du, Ahmed Hassan, Adam Fourney, Robert Sim, Paul Bennett, Claire Cardie
Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, Pages 551-556. December 2020. pdf

Proactive Suggestion Generation: Data and Methods for Stepwise Task Assistance, Elnaz Nouri, Robert Sim, Adam Fourney, Ryen W White
Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2020), Pages 1585-1588, July 2020. pdf

Conversations with Documents: An Exploration of Document-Centered Assistance, Maartje ter Hoeve, Robert Sim, Elnaz Nouri, Adam Fourney, Maarten de Rijke and Ryen W. White
5th ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR 2020), Pages: 43-52 Vancouver, Canada, March 2020. pdf

Step-wise Recommendation for Complex Task Support, Elnaz Nouri, Robert Sim, Adam Fourney, and Ryen W. White
5th ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR 2020), Pages: 203-212 Vancouver, Canada, March 2020. pdf

Toward Activity Discovery in the Personal Web, Tara Safavi, Adam Fourney, Robert Sim, Marcin Juraszek, Shane Williams, Ned Friend, Danai Koutra, Paul N. Bennett.
13th Annual International ACM WSDM Conference on Web Search and Data Mining (WSDM 2020), Houston, TX. pdf

Task Completion Detection: A Study in the Context of Intelligent Systems , Ryen W. White, Ahmed Hassan Awadallah and Robert Sim
42nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019), Pages: N/A Paris, France, July 2019. Pages: 405-414. pdf

Domain Adaptation for Commitment Detection in Email , Hosein Azarbonyad, Robert Sim, and Ryen W. White
12th Annual International ACM WSDM Conference on Web Search and Data Mining (WSDM 2019), Pages: N/A Melbourne, Australia, February 2019 pdf

Supporting Complex Tasks Using Multiple Devices, Elnaz Nouri, Adam Fourney, Robert Sim, and Ryen W. White
Task Intelligence Workshop at WSDM (TI@WSDM 2019) Melbourne, Australia, February 2019 pdf

Autonomous vision-based exploration and mapping using hybrid maps and Rao-Blackwellised particle filters, Robert Sim and James J. Little.
Proceedings of the IEEE/RSJ Conference on Intelligent Robots and Systems (IROS), Beijing, 2006. [ pre-print: pdf ] Nominated for best paper.

Design and analysis of a framework for real-time vision-based SLAM using Rao-Blackwellised particle filters, Robert Sim, Pantelis Elinas, Matt Griffin, Alex Shyr, and James J. Little
Proceedings of the 3rd Canadian Conference on Computer and Robotic Vision (CRV 2006), Quebec City, QC, 9 pages, 2006. Best Robotics Paper [ pdf ]

σSLAM: Stereo Vision SLAM Using the Rao-Blackwellised Particle Filter and a Novel Mixture Proposal Distribution, Pantelis Elinas, Robert Sim, and James J. Little
Proceedings of the IEEE International Conference on Robotics and Automation, Orlando, FL, 7 pages, 2006. [ pdf ]

Scalable real-time vision-based SLAM for planetary rovers, Robert Sim, Matt Griffin, Alex Shyr, and James J. Little
Proceedings of the IEEE IROS Workshop on Robot Vision for Space Applications, Edmonton, AB, 6 pages, 2005. [ pdf ]

Stabilizing Information-Driven Exploration for Bearings-Only SLAM Using Range Gating, Robert Sim
Proceedings of Intelligent Robots and Systems (IROS), Edmonton, AB, 6 pages, 2005. [ pdf ]

Vision-based SLAM using the Rao-Blackwellised Particle Filter, Robert Sim, Pantelis Elinas, Matt Griffin, and James J. Little
IJCAI Workshop on Reasoning with Uncertainty in Robotics (RUR), Edinburgh, Scotland, 8 pages, 2005. [ pdf ]

Stable Exploration for Bearings-only SLAM, Robert Sim
Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Barcelona, Spain, 6 pages. 2005. [ pdf ]

Global A-Optimal Robot Exploration in SLAM, Robert Sim and Nicholas Roy
Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), Barcelona, Spain, 6 pages. 2005. [ pdf ]

Learning Generative Models of Invariant Features, Robert Sim and Gregory Dudek
Proceedings of Intelligent Robots and Systems (IROS), Sendai, Japan. 8 pages. 2004. [ pdf ]

Landmark Selection for Vision-Based Navigation, Pablo Sala, Robert Sim, Ali Shokoufandeh and Sven J. Dickinson
Proceedings of Intelligent Robots and Systems (IROS), Sendai, Japan. 8 pages. 2004. [ pdf ]

AQUA: an aquatic walking robot, C. Georgiades, A. German, A. Hogue, H. Liu, C. Prahacs, A. Ripsman, R. Sim, L. A. Torres, P. Zhang, M. Buehler, G. Dudek, M. Jenkin and E. Milios
Proceedings of Intelligent Robots and Systems (IROS), Sendai, Japan. 8 pages. 2004.

Self-Organizing Visual Maps, Robert Sim and Gregory Dudek
Proceedings of the National Conference on Artificial Intelligence (AAAI), San Jose, CA. 6 pages. 2004. [ pdf ]

Online Control Policy Optimization for Minimizing Map Uncertainty during Exploration, Robert Sim, Gregory Dudek and Nicholas Roy
Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), New Orleans, LA. 6 pages. 2004. [ pdf ]

Effective Exploration Strategies for the Construction of Visual Maps, Robert Sim and Gregory Dudek
Proceedings of the IEEE/RSJ Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV. 8 pages. 2003. [ pdf ]

Examining Exploratory Trajectories for Minimizing Map Uncertainty, Robert Sim and Gregory Dudek
Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI) workshop on Reasoning with Uncertainty in Robotics (RUR), Acapulco, Mexico. pp 69--76. 2003. [ pdf ]

Comparing image-based localization methods, Robert Sim and Gregory Dudek
Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI), Acapulco, Mexico, 2003, 3 pages. [ pdf ]

Self-Organizing Visual Maps, Robert Sim and Gregory Dudek
Workshop on Advances in Machine Learning (presentation, no published proceedings) Montreal, Quebec. 6 pages. 2003. [ pdf ]

RoboDaemon - A device independent, network-oriented, modular mobile robot controller, Gregory Dudek and Robert Sim
Proceedings of the IEEE Conference on Robotics and Automation (ICRA), Taipei, Taiwan, 2003, 7 pages. [ pdf (485kb)]

Learning Generative Models of Scene Features, Robert Sim and Gregory Dudek
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Hawaii, 2001, 7 pages. [ postscript (915kb)] [ pdf (525kb)]

Collaborative Exploration for the Construction of Visual Maps, Ioannis Rekleitis, Robert Sim, Gregory Dudek and Evangelos Milios
Proceedings of IEEE/RSJ Conference on Intelligent Robots and Systems (IROS), Hawaii, 2001, 6 pages. [ postscript (1060kb)] [ pdf (575kb)]

Collaborative Exploration for Map Construction, Ioannis Rekleitis, Robert Sim, Gregory Dudek and Evangelos Milios
International Symposium on Computational Intelligence in Robotics and Automation (CIRA), Banff, Alberta, 2001, 6 pages. [ postscript (829kb)]

Learning Landmarks for Robot Localization, Robert Sim and Gregory Dudek
AAAI-2000 Doctoral Consortium, Austin, Texas, July 2000, 2 pages. [ postscript (82kb)]

Learning and Evaluating Visual Features for Pose Estimation, Robert Sim and Gregory Dudek
Proceedings of International Conference on Computer Vision, Kerkyra, Greece, September 1999, 6 pages. Paper and Poster. [ postscript (450kb)]

Learning Environmental Features for Pose Estimation, Robert Sim and Gregory Dudek
CVPR Workshop on Perception for Mobile Agents, Ft. Collins, CO, June 1999, 8 pages. [ postscript (669kb)]

Learning Visual Landmarks for Pose Estimation, Robert Sim and Gregory Dudek
Proceedings of International Conference on Robotics and Automation (ICRA), Detroit, MI, May 1999, 7 pages. [ postscript (433kb)]

Mobile robot localization from learned landmarks, Robert Sim and Gregory Dudek
Proceedings of IEEE/RSJ Conference on Intelligent Robots and Systems (IROS), Victoria, BC, Oct 1998, 6 pages. [ postscript (223 kB)]

invited publications

Visual Landmarks for Pose Estimation, Robert Sim and Gregory Dudek
Canadian Artificial Intelligence, No. 43, Spring 1999 pp13-17

abstracts

On Visual Maps and their Automatic Construction, Robert Sim and Gregory Dudek
Precarn/IRIS Conference, June 2003.

Position Estimation from Learned Landmarks, Robert Sim and Gregory Dudek
Second Prize Poster, Precarn/IRIS Conference, June 1998.

Navigating by the Stars: A landmark-based method for Mobile Robot Localization, Robert Sim and Gregory Dudek
Poster. Precarn/IRIS Conference, June 1997.

Autonomous Exploration: An integrated Systems Approach, Bolduc, M. Bourque, E. Dudek, G. Roy, N. and Sim, R.
Proc. National Conference on Artificial Intelligence, July 1996, pp.779-780, MIT Press.

non-refereed articles and technical reports

Training Data Leakage Analysis in Language Models, Huseyin A. Inan, Osman Ramadan, Lukas Wutschitz, Daniel Jones, Victor Rühle, James Withers, Robert Sim
, Microsoft, 2021 [ arxiv ]

Self-Organizing Visual Maps, Robert Sim and Gregory Dudek
Technical Report, Centre for Intelligent Machines, McGill University, 16 pages. June, 2003 [ pdf ]

Comparing Attention Operators for Learning Landmarks, Robert Sim, Sandra Polifroni and Gregory Dudek
Technical Report, Centre for Intelligent Machines, McGill University, 16 pages. October, 2001 [ pdf (315 kB)]

Bayesian Exploration for Mobile Robots, Robert Sim
Technical Report, CIM-03-02, Centre for Intelligent Machines, McGill University, 23 pages. June, 2000 [ pdf (330 kB)]

Mobile robot localization from learned landmarks, Robert Sim
Technical Report, CIM-98-03, Centre for Intelligent Machines, McGill University, 66 pages. November, 1998 [ postscript (2.5 MB)]

Robert Sim, Last modified: 4 Feb 2020