I'm currently a Principal Research Manager at Microsoft Research. I'm currently exploring problems in privacy-preserving machine learning, conversational task assistance, and instrumenting group interactions (meetings) towards building a better understanding of the relationships between interpersonal behaviors and meeting effectiveness.
From about 2008 to 2017 I was an applied researcher and occasional research manager in Microsoft's Safety Platform, where we did a lot of interesting work on protecting users online- spam prevention, family safety, account protection, fraud and scam prevention and phishing and malware detection.
Message from a past life (circa 2005): My research explores problems in mobile robot localization and environment learning using vision and range sensors. In short, I'm looking to answer questions like: Given a map, and a sensor reading, what is the most plausible place I might be in? How can I move to improve my confidence about where I am? What are the best places to move to to build a map? What is the best representation for a map? How can attention be used to simplify the map-building process?
Previously I was a Senior Robotic Vision Scientist at Braintech, Inc.
For a while in 2006 I was a vision researcher at Microsoft adCenter Labs.
From Jan 2004-July 2006 I was at UBC, working as a postdoc with Jim Little and David Lowe.
In January, 2004, I defended my Ph.D. Thesis, entitled On Visual Maps and their Automatic Construction.
In the fall of 2003, I worked at UofT with Sven Dickinson.
In 2000, I submitted my Ph.D. Proposal on Bayesian Exploration under the supervision of Prof. Greg Dudek at the Centre for Intelligent Machines, McGill University.
My Master's thesis is entitled "Mobile Robot Localization from Learned Landmarks". You can browse my thesis online here.
Check out the mobile robotics lab (my former lab) for more info.
My (out-of-date) BibTeX file is here.
Multi-Device Digital Assistance, Ryen W. White, Adam Fourney, Allen Herring, Paul Bennett, Niru Chandrasekaran, Robert Sim, Elnaz Nouri, Mark Encarnacion, and Nalin Singal. Communications of the ACM (CACM), Vol: 62, Issue: 10, Pages: 28-31. [pdf]
Autonomous vision-based robotic exploration and mapping using hybrid maps and particle filters, R. Sim and James J. Little, Image and Vision Computing, Volume 27 Issue 1-2, January, 2009 Pages 167-177. ACM Link
A study of the Rao-Blackwellised particle filter for efficient and accurate vision-based SLAM, R. Sim, P. Elinas and J. J. Little, International Journal of Computer Vision/International Journal of Robotics Research Special Joint Issue on Vision in Robotics,vol 74, no 3, pp 303-318, Kluwer, Sept 2007
Landmark Selection for Vision-Based Navigation, P. L. Sala, R. Sim, A. Shokoufandeh and S. J. Dickinson. IEEE Transactions on Robotics, vol 22, no 2, pp 334-349. IEEE Press. April 2006.[preprint]
Learning generative models of scene features, R. Sim and G. Dudek, International Journal of Computer Vision vol 60, no 1, pp 45-61, Kluwer, 2004. Pre-press version: [pdf] [Kluwer on-line ]
Learning environmental features for pose estimation, R. Sim and G. Dudek, Image and Vision Computing, vol 19, num 11, pp 733-739, Elsevier Press, 2001. [Reprints available on request] [ postscript (650kb)]
Computer support for meetings,
Robert Alexander Sim, Marcello MENDES HASEGAWA, Ryen William White, Mudit Jain, Tomer Hermelin, Adi GERZI ROSENTHAL, Sagi Hilleli
US11113672
Brokered Authentication with Risk Sharing, Robert A Sim, Akash A Shah, Jisheng Liang
US10652282
Empathetic personal virtual digital assistant, Nirupama Chandrasekaran, Robert A Sim, Ryen W. White, Nikrouz Ghotbi
US10522143
Risk assessment modeling,
Luke Abrams, David J. Steeves, Robert Alexander Sim, Pui-Yin Winfred Wong, Harry Simon Katz, Aaron Small, Dana Scott Kaufman,
Adrian Kreuziger, Mark A. Nikiel, Laurentiu Bogdan Cristofor, Alexa Lynn Keizur, Collin Tibbetts, Charles Hayden
US9779236B2
Chunk-Based File Acquisition And File Reputation Evaluation,
Robert Alexander Sim, Christian Seifert,Anthony Penta, Elliott Jeb Haber, Tomasz Kasperkiewicz
US9398036
Generating stimuli for use in soliciting grounded linguistic information,
William B. Dolan, Christopher I. Charla, Christopher B. Quirk, Christopher J. Brockett, Noelle M. Sophy, Nicole Beaudry,
Vikram Reddy Dendi, Pallavi Choudhury, Scott T. Laufer, Robert A. Sim, Thomas E. Woolsey, David Molnar
US9116880B2
Using the ability to speak as a human interactive proof,
Chad Mills, Robert Sim, Scott Laufer, Sung Chung
US9390245B2
Intelligent Information Display, Li Li, Robert A. Sim, Tarek Najm, Ying Li, Yi Chen.
US20080262909A1
On Visual Maps and their Automatic Construction, Robert Sim
Ph.D. Thesis, 170 pages.
[
pdf (7.0Mb)]
Mobile robot localization from learned landmarks, Robert Sim
Master's Thesis, McGill University, 66 pages. July, 1998
[
postscript (2.5 MB)]
To Boldly Go: Bayesian Exploration for Mobile Robots, Robert Sim
Ph.D. Proposal, 24 pages.
[
postscript (278kb)]
Understanding Questions that Arise When Working with Business Documents,
Farnaz Jahanbakhsh, Elnaz Nouri, Robert Sim, Ryen W. White, Adam Fourney
CSCW 2022. [arxiv]
UserIdentifier: Implicit User Representations for Simple and Effective Personalized Sentiment Analysis,
Fatemehsadat Mireshghallah, Vaishnavi Shrivastava, Milad Shokouhi, Taylor Berg-Kirkpatrick, Robert Sim, Dimitrios Dimitriadis
Fl4NLP @ ACL 2022. [arxiv]
Membership Inference Attacks Against NLP Classification Models,
Virat Shejwalkar, Huseyin A Inan, Amir Houmansadr, Robert Sim
NeurIPS 2021 Workshop Privacy in Machine Learning. [pdf]"]
On Privacy and Confidentiality of Communications in Organizational Graphs,
Masoumeh Shafieinejad, Huseyin Inan, Marcello Hasegawa, Robert Sim
Distributed and Private Machine Learning (DPML) Workshop, ICLR 2021.
[pdf]
Stereotyping Norwegian salmon: an inventory of pitfalls in fairness benchmark datasets,
Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, Hanna Wallach
Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the
11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) pp1004-1015.
[pdf]
Privacy Regularization: Joint Privacy-Utility Optimization in Language Models,
Fatemehsadat Mireshghallah, Huseyin A. Inan, Marcello Hasegawa, Victor Rühle, Taylor Berg-Kirkpatrick, Robert Sim
2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL 2021).
[pdf, arxiv]
MeetingCoach: An Intelligent Dashboard for Supporting Effective and Inclusive Meetings,
Samiha Samrose, Daniel McDuff, Robert Sim, Jina Suh, Kael Rowan, Javier Hernandez, Sean Rintel, Kevin Moynihan, Mary Czerwinski
CHI 2021.
pdf
Leveraging Structured Metadata for Improving Question Answering on the Web,
Xinya Du, Ahmed Hassan, Adam Fourney, Robert Sim, Paul Bennett, Claire Cardie
Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, Pages 551-556. December 2020.
pdf
Proactive Suggestion Generation: Data and Methods for Stepwise Task Assistance,
Elnaz Nouri, Robert Sim, Adam Fourney, Ryen W White
Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2020), Pages 1585-1588, July 2020.
pdf
Conversations with Documents: An Exploration of Document-Centered Assistance,
Maartje ter Hoeve, Robert Sim, Elnaz Nouri, Adam Fourney, Maarten de Rijke and Ryen W. White
5th ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR 2020), Pages: 43-52
Vancouver, Canada, March 2020. pdf
Step-wise Recommendation for Complex Task Support,
Elnaz Nouri, Robert Sim, Adam Fourney, and Ryen W. White
5th ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR 2020), Pages: 203-212
Vancouver, Canada, March 2020. pdf
Toward Activity Discovery in the Personal Web, Tara Safavi, Adam Fourney, Robert Sim, Marcin Juraszek, Shane Williams, Ned Friend, Danai Koutra, Paul N. Bennett.
13th Annual International ACM WSDM Conference on Web Search and Data Mining (WSDM 2020), Houston, TX. pdf
Task Completion Detection: A Study in the Context of Intelligent Systems ,
Ryen W. White, Ahmed Hassan Awadallah and Robert Sim
42nd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 2019), Pages: N/A
Paris, France, July 2019. Pages: 405-414. pdf
Domain Adaptation for Commitment Detection in Email ,
Hosein Azarbonyad, Robert Sim, and Ryen W. White
12th Annual International ACM WSDM Conference on Web Search and Data Mining (WSDM 2019), Pages: N/A
Melbourne, Australia, February 2019 pdf
Supporting Complex Tasks Using Multiple Devices,
Elnaz Nouri, Adam Fourney, Robert Sim, and Ryen W. White
Task Intelligence Workshop at WSDM (TI@WSDM 2019)
Melbourne, Australia, February 2019 pdf
Autonomous vision-based exploration and mapping using
hybrid maps and Rao-Blackwellised particle filters,
Robert Sim and James J. Little.
Proceedings of the IEEE/RSJ Conference on Intelligent Robots and
Systems (IROS), Beijing, 2006. [ pre-print: pdf ] Nominated for best paper.
Design and analysis of a framework for real-time vision-based SLAM
using
Rao-Blackwellised particle filters, Robert Sim, Pantelis Elinas,
Matt Griffin, Alex Shyr, and James J. Little
Proceedings of the 3rd Canadian Conference on Computer and Robotic
Vision (CRV 2006), Quebec City, QC, 9 pages, 2006. Best Robotics
Paper [ pdf ]
σSLAM: Stereo Vision SLAM Using the
Rao-Blackwellised Particle Filter and a Novel
Mixture Proposal Distribution, Pantelis Elinas, Robert Sim, and James J. Little
Proceedings of the IEEE International Conference on Robotics and
Automation, Orlando, FL, 7 pages, 2006. [ pdf ]
Scalable real-time vision-based SLAM for planetary
rovers, Robert Sim, Matt Griffin, Alex Shyr, and James J. Little
Proceedings of the IEEE IROS Workshop on Robot Vision for Space
Applications, Edmonton, AB, 6 pages, 2005. [ pdf ]
Stabilizing Information-Driven Exploration for Bearings-Only SLAM Using Range Gating,
Robert Sim
Proceedings of Intelligent Robots and Systems
(IROS), Edmonton, AB, 6 pages,
2005. [ pdf ]
Vision-based SLAM using the Rao-Blackwellised Particle Filter,
Robert Sim, Pantelis Elinas, Matt Griffin, and James J. Little
IJCAI Workshop on Reasoning with Uncertainty in Robotics (RUR), Edinburgh, Scotland, 8 pages,
2005. [ pdf ]
Stable Exploration for Bearings-only SLAM, Robert Sim
Proceedings of the IEEE International Conference on Robotics and
Automation (ICRA), Barcelona, Spain, 6 pages. 2005. [ pdf ]
Global A-Optimal Robot Exploration in SLAM, Robert Sim and
Nicholas Roy
Proceedings of the IEEE International Conference on Robotics and
Automation (ICRA), Barcelona, Spain, 6 pages. 2005. [ pdf ]
Learning Generative Models of Invariant Features, Robert Sim and Gregory Dudek
Proceedings of Intelligent Robots and Systems (IROS), Sendai, Japan. 8 pages. 2004.
[ pdf ]
Landmark Selection for Vision-Based Navigation, Pablo Sala,
Robert Sim, Ali Shokoufandeh and Sven J. Dickinson
Proceedings of Intelligent Robots and Systems (IROS), Sendai, Japan. 8 pages. 2004.
[ pdf ]
AQUA: an aquatic walking robot, C. Georgiades, A. German, A. Hogue, H. Liu, C. Prahacs, A. Ripsman, R. Sim, L. A. Torres, P. Zhang, M. Buehler, G. Dudek, M. Jenkin and E. Milios
Proceedings of Intelligent Robots and Systems (IROS), Sendai, Japan. 8 pages. 2004.
Self-Organizing Visual Maps, Robert Sim and Gregory Dudek
Proceedings of the National Conference on Artificial Intelligence (AAAI), San
Jose, CA. 6 pages. 2004.
[ pdf ]
Online Control Policy Optimization for Minimizing Map Uncertainty
during Exploration, Robert Sim, Gregory Dudek and Nicholas Roy
Proceedings of the IEEE International Conference on Robotics and Automation
(ICRA), New Orleans, LA. 6 pages. 2004.
[ pdf ]
Effective Exploration Strategies for the Construction of Visual Maps, Robert Sim and Gregory Dudek
Proceedings of the IEEE/RSJ Conference on Intelligent Robots and Systems
(IROS), Las Vegas, NV. 8 pages. 2003.
[ pdf ]
Examining Exploratory Trajectories for Minimizing Map Uncertainty, Robert Sim and Gregory Dudek
Proceedings of the International Joint Conference
on Artificial Intelligence (IJCAI) workshop on Reasoning with Uncertainty in
Robotics (RUR), Acapulco, Mexico. pp 69--76. 2003.
[ pdf ]
Comparing image-based localization methods, Robert Sim and Gregory Dudek
Proceedings of the Eighteenth International Joint Conference
on Artificial Intelligence (IJCAI), Acapulco, Mexico, 2003, 3 pages.
[ pdf ]
Self-Organizing Visual Maps, Robert Sim and Gregory Dudek
Workshop on Advances in Machine Learning (presentation, no published
proceedings) Montreal, Quebec. 6 pages. 2003. [ pdf ]
RoboDaemon - A device independent, network-oriented, modular mobile robot
controller, Gregory Dudek and Robert Sim
Proceedings of the IEEE Conference on Robotics and Automation
(ICRA), Taipei, Taiwan, 2003, 7 pages.
[
pdf (485kb)]
Learning Generative Models of Scene Features, Robert Sim and Gregory Dudek
Proceedings of the IEEE Conference on Computer Vision and Pattern
Recognition (CVPR), Hawaii, 2001, 7 pages.
[
postscript (915kb)] [
pdf (525kb)]
Collaborative Exploration for the Construction of Visual Maps, Ioannis Rekleitis, Robert Sim, Gregory Dudek and Evangelos Milios
Proceedings of IEEE/RSJ Conference on Intelligent Robots and Systems
(IROS), Hawaii, 2001, 6 pages.
[
postscript (1060kb)] [
pdf (575kb)]
Collaborative Exploration for Map Construction, Ioannis Rekleitis, Robert Sim, Gregory Dudek and Evangelos Milios
International Symposium on Computational Intelligence in Robotics and
Automation (CIRA), Banff, Alberta, 2001, 6 pages.
[
postscript (829kb)]
Learning Landmarks for Robot Localization, Robert Sim and Gregory Dudek
AAAI-2000 Doctoral Consortium,
Austin, Texas, July 2000, 2 pages.
[
postscript (82kb)]
Learning and Evaluating Visual Features for Pose Estimation, Robert Sim and Gregory Dudek
Proceedings of International Conference on Computer Vision,
Kerkyra, Greece, September 1999, 6 pages. Paper and Poster.
[
postscript (450kb)]
Learning Environmental Features for Pose Estimation, Robert Sim and Gregory Dudek
CVPR Workshop on Perception for Mobile Agents,
Ft. Collins, CO, June 1999, 8 pages.
[
postscript (669kb)]
Learning Visual Landmarks for Pose Estimation, Robert Sim and Gregory Dudek
Proceedings of International Conference on Robotics and Automation
(ICRA), Detroit, MI, May 1999, 7 pages.
[
postscript (433kb)]
Mobile robot localization from learned landmarks, Robert Sim and Gregory Dudek
Proceedings of IEEE/RSJ Conference on Intelligent Robots and Systems
(IROS), Victoria, BC, Oct 1998, 6 pages.
[
postscript (223 kB)]
Visual Landmarks for Pose Estimation, Robert Sim and Gregory Dudek
Canadian Artificial Intelligence, No. 43, Spring 1999 pp13-17
On Visual Maps and their Automatic Construction, Robert Sim and Gregory Dudek
Precarn/IRIS Conference, June 2003.
Position Estimation from Learned Landmarks, Robert Sim and Gregory Dudek
Second Prize Poster, Precarn/IRIS Conference, June 1998.
Navigating by the Stars: A landmark-based method for Mobile Robot Localization, Robert Sim and Gregory Dudek
Poster. Precarn/IRIS Conference, June 1997.
Autonomous Exploration: An integrated Systems Approach, Bolduc, M. Bourque, E. Dudek, G. Roy, N. and Sim, R.
Proc. National Conference on Artificial Intelligence, July 1996, pp.779-780, MIT Press.
FLUTE: A Scalable, Extensible Framework for High-Performance Federated Learning Simulations,
Dimitrios Dimitriadis, Mirian Hipolito Garcia, Daniel Madrigal Diaz, Andre Manoel, Robert Sim,
Microsoft, 2022. [arxiv]
Training Data Leakage Analysis in Language Models,
Huseyin A. Inan, Osman Ramadan, Lukas Wutschitz, Daniel Jones, Victor Rühle, James Withers, Robert Sim,
Microsoft, 2021
[ arxiv ]
Self-Organizing Visual Maps, Robert Sim and Gregory Dudek
Technical Report, Centre for Intelligent Machines, McGill
University, 16 pages. June, 2003
[
pdf ]
Comparing Attention Operators for Learning Landmarks, Robert Sim, Sandra
Polifroni and Gregory Dudek
Technical Report, Centre for Intelligent Machines, McGill
University, 16 pages. October, 2001
[
pdf (315 kB)]
Bayesian Exploration for Mobile Robots, Robert Sim
Technical Report, CIM-03-02, Centre for Intelligent Machines, McGill
University, 23 pages. June, 2000
[
pdf (330 kB)]
Mobile robot localization from learned landmarks, Robert Sim
Technical Report, CIM-98-03, Centre for Intelligent Machines, McGill University, 66 pages. November, 1998
[
postscript (2.5 MB)]