Research Interests

  • Video and Audio Understanding
  • Recommendation Systems
  • Collaborative Filtering
  • Statistical Machine Learning

Publications

Google Scholar | DBLP | Semantic Scholar
  • Book Chapters
    1. Joonseok Lee. Recommendation Systems: An Industrial Application of Network Big Data for Computational Intelligence, Big Data and Computational Intelligence in Networking, Yulei Wu, Fei Hu, Geyong Min, Albert Y. Zomaya (ed.), CRC Press, ISBN: 9781498784863, Chapter 11, 2017. [link] [book]
  • Referred Journals
    1. Sangho Suh, Sungbok Shin, Joonseok Lee, Chandan K. Reddy, Jaegul Choo. Localized User-Driven Topic Discovery via Boosted Ensemble of Nonnegative Matrix Factorization, Knowledge and Information Systems (KAIS), 2018. [pdf]
    2. Joonseok Lee, Seungyeon Kim, Guy Lebanon, Yoram Singer, Samy Bengio. LLORMA: Local Low-Rank Matrix Approximation, Journal of Machine Learning Research (JMLR) 17(15):1-24, 2016. [pdf] [talk by Lebanon]
    3. Joonseok Lee, Hanggjun Cho, Robert Ian (Bob) McKay. A Rapid Screening and Testing Protocol for Keyboard Layout Speed Comparison, IEEE Transactions on Human-Machine Systems, vol.PP, no.99, pp.1-14, 2014. [pdf]
    4. Joonseok Lee, Mingxuan Sun, Guy Lebanon. PREA: Personalized Recommendation Algorithms Toolkit, Journal of Machine Learning Research (JMLR) 13:2699-2703, 2012. [pdf] [code]
  • Referred Conferences
    1. Jinyeong Chae, Donghwa Kim, Kwanseok Kim, Doyeon Lee, Sangho Lee, Seongsu Ha, Jonghwan Mun, Wooyoung Kang, Byungseok Roh, Joonseok Lee. Towards a Complete Benchmark on Video Moment Localization, Proceedings of the 27th International Conference on Artificial Intelligence and Statistics (AISTATS), 2024. [pdf]
    2. Suhwan Choi, Myeongho Jeon, Yeonjung Hwang, Jeonglyul Oh, Sungjun Lim, Joonseok Lee, Myungjoo Kang. Dictionary Contrastive Forward Learning via Adaptive Label Embeddings, Proceedings of the 12th International Conference on Learning Representations (ICLR), 2024.
    3. Kun Su, Judith Li, Qingqing Huang, Dima Kuzmin, Joonseok Lee, Chris Donahue, Fei Sha, Aren Jansen, Yu Wang, Mauro Verzetti, Timo Denk. V2Meow: Meowing to the Visual Beat via Video-to-Music Generation, Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI), 2024. [pdf]
    4. Eunyi Lyou, Doyeon Lee, Jooeun Kim, Joonseok Lee. Modality-Aware Representation Learning for Zero-shot Sketch-based Image Retrieval, Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024. [pdf]
    5. Seo Young Lee, Joonseok Lee. PoseDiff: Pose-conditioned Diffusion for Sparse View Unbounded Scene Synthesis, Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024. [pdf]
    6. YunSeong Cho, Chanwoo Kim, Hoseong Cho, Yunhoe Ku, Eunseo Kim, Muhammadjon Boboev, Joonseok Lee, Seungryul Baek. RMFER: Semi-supervised Contrastive Learning for Facial Expression Recognition with Reaction Mashup Video, Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2024. Best Paper Finalist [link] [pdf]
    7. Jinhwan Sul, Jihoon Han, Joonseok Lee. Mr. HiSum: A Large-scale Dataset and Benchmark for Video Highlight Detection and Summarization, Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS), 2023. [pdf]
    8. Jiyoung Lee, Seungho Kim, Seunghyun Won, Joonseok Lee, Marzyeh Ghassemi, James Thorne, Jaeseok Choi, O-Kil Kwon, Edward Choi. VisAlign: Dataset for Measuring the Degree of Alignment between AI and Humans in Visual Perception, Proceedings of the 37th Conference on Neural Information Processing Systems (NeurIPS), 2023. [pdf]
    9. Sungchan Park, Eunyi Lyou, Inhoe Lee, Joonseok Lee. Towards Robust and Smooth 3D Multi-Person Pose Estimation from Monocular Videos in the Wild, Proceedings of the 19th IEEE/CVF International Conference on Computer Vision (ICCV), 2023. [pdf]
    10. Myeongho Jeon, Myungjoo Kang, Joonseok Lee. A Unified Framework for Robustness on Diverse Sampling Errors, Proceedings of the 19th IEEE/CVF International Conference on Computer Vision (ICCV), 2023. [pdf]
    11. Seunghoon Yi, Youngwoo Cho, Jinhwan Sul, Seung Woo Ko, Soo Kyung Kim, Jaegul Choo, Hongkee Yoon, Joonseok Lee. Towards Physically Reliable Molecular Representation Learning, Proceedings of the 39th Conference on Uncertainty in Artificial Intelligence (UAI), 2023. [pdf] [talk]
    12. Woncheol Shin, Gyubok Lee, Jiyoung Lee, Eunyi Lyou, Joonseok Lee, Edward Choi. Exploration into Translation-Equivariant Image Quantization, Proceedings of the 48th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2023. Top 3% Paper [pdf]
    13. Daeun Kyung, Kyungmin Jo, Jaegul Choo, Joonseok Lee, Edward Choi. Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays, Proceedings of the 48th IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2023. Best Student Paper Award [pdf]
    14. Sangho Lee, Seoyoung Lee, Joonseok Lee. Learning to Wear: Details-Preserved Virtual Try-on via Disentangling Clothes and Wearer, Proceedings of the 33rd British Machine Vision Conference (BMVC), 2022. [pdf]
    15. Yeji Song, Chaerin Kong, Seoyoung Lee, Nojun Kwak, Joonseok Lee. Towards Efficient Neural Scene Graphs by Learning Consistency Fields, Proceedings of the 33rd British Machine Vision Conference (BMVC), 2022. [pdf]
    16. Hyungyung Lee, Sungjin Park, Joonseok Lee, Edward Choi. Unconditional Image-Text Pair Generation with Multimodal Cross Quantizer, Proceedings of the 33rd British Machine Vision Conference (BMVC), 2022. [pdf]
    17. Jonghwan Mun, Minchul Shin, Gunsoo Han, Sangho Lee, Seongsu Ha, Joonseok Lee, Eun-Sol Kim. BaSSL: Boundary-aware Self-Supervised Learning for Video Scene Segmentation, Proceedings of the 16th Asian Conference on Computer Vision (ACCV), 2022. [pdf]
    18. Qingqing Huang, Aren Jansen, Joonseok Lee, Ravi Ganti, Judith Li, Dan Ellis. MuLan: A Joint Embedding of Music Audio and Natural Language, Proceedings of the 23rd International Society for Music Information Retrieval Conference (ISMIR), 2022. [pdf]
    19. Jae-woong Lee, Seongmin Park, Joonseok Lee, Jongwuk Lee. Bilateral Self-unbiased Learning from Biased Implicit Feedback, Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), 2022. [pdf]
    20. Myeongho Jeon, Daekyung Kim, Woochul Lee, Myungjoo Kang, Joonseok Lee. A Conservative Approach for Unbiased Learning on Unknown Biases, Proceedings of the 38th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022. [pdf]
    21. Minjin Choi, Jinhong Kim, Joonseok Lee, Hyunjung Shim, Jongwuk Lee. S-Walk: Accurate and Scalable Session-based Recommendation with Random Walks, Proceedings of the 15th ACM International Conference on Web Search and Data Mining (WSDM), 2022. [pdf]
    22. Kangyeol Kim, Sunghyun Park, Junsoo Lee, Joonseok Lee, Sookyung Kim, Jaegul Choo, Edward Choi. Continuous-Time Video Generation via Learning Motion Dynamics with Neural ODE, Proceedings of the 32nd British Machine Vision Conference (BMVC), 2021. [pdf]
    23. Minjin Choi, Jinhong Kim, Joonseok Lee, Hyunjung Shim, Jongwuk Lee. Session-aware Linear Item-Item Models for Session-based Recommendation, Proceedings of the 31st ACM Web Conference (WWW), 2021. [pdf]
    24. Minjin Choi, Yoonki Jeong, Joonseok Lee, Jongwuk Lee. Local Collaborative Autoencoders, Proceedings of the 14th ACM International Conference on Web Search and Data Mining (WSDM), 2021. [pdf]
    25. Sunghyun Park, Kangyeol Kim, Junsoo Lee, Jaegul Choo, Joonseok Lee, Sookyung Kim, Edward Choi. Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Differential Equation, Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI), 2021. [pdf]
    26. Hyodong Lee, Joonseok Lee, Joe Ng, Paul Natsev. Large Scale Video Representation Learning via Relational Graph Clustering, Proceedings of the 36th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020. [pdf] [video]
    27. Inioluwa Deborah Raji, Timnit Gebru, Margaret Mitchell, Joy Buolamwini, Joonseok Lee, Emily Denton. Saving Face: Investigating the Ethical Concerns of Facial Recognition Auditing, Proceedings of the 3rd AAAI/ACM Conference on Artificial Intelligence, Ethics, and Society (AIES), 2020. [pdf]
    28. Sookyung Kim, Sunghyun Park, Sunghyo Chung, Joonseok Lee, Yunsung Lee, Hyojin Kim, Mr Prabhat, Jaegul Choo. Learning to Focus and Track Extreme Climate Events, Proceedings of the 30th British Machine Vision Conference (BMVC), 2019. [pdf] [supplementary]
    29. Seong Jae Hwang, Joonseok Lee, Balakrishnan Varadarajan, Ariel Gordon, Zheng Xu, Paul Natsev. Large-Scale Training Framework for Video Annotation, Proceedings of the 25th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2019. [pdf] [talk] [video]
    30. Sami Abu-El-Haija, Amol Kapoor, Bryan Perozzi, Joonseok Lee. N-GCN: Multi-scale Graph Convolution for Semi-supervised Node Classification, Proceedings of the 35th Conference on Uncertainty in Artificial Intelligence (UAI), 2019. [pdf] [talk]
    31. Sookyung Kim, Hyojin Kim, Joonseok Lee, Sangwoong Yoon, Samira Ebrahimi Kahou, Karthik Kashinath, Mr. Prabhat. Deep-Hurricane-Tracker: Tracking and Forecasting Extreme Climate Events, Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), 2019. [pdf]
    32. Sookyung Kim, Sangwoong Yoon, Joonseok Lee, Samira Ebrahimi Kahou, Hyojin Kim, Karthik Kashinath, Mr. Prabhat. Deep-Hurricane-Tracker: Tracking Extreme Climate Events, Climate Informatics, 2018. [pdf]
    33. Joonseok Lee, Paul Natsev, Walter Reade, Rahul Sukthankar, George Toderici. The 2nd YouTube-8M Large-Scale Video Understanding Challenge, Proceedings of the 15th Euoprean Conference on Computer Vision (ECCV), 2018. [pdf]
    34. Joonseok Lee, Sami Abu-El-Haija, Balakrishnan Varadarajan, Paul Natsev. Collaborative Deep Metric Learning for Video Understanding, Proceedings of the 24th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2018. [pdf] [video]
    35. Joonseok Lee, Sami Abu-El-Haija. Large-Scale Content-Only Video Recommendation, Proceedings of the IEEE International Conference on Computer Vision (ICCV), CEFRL Workshop, pp.987-995, 2017. [pdf]
    36. Sangho Suh, Jaegul Choo, Joonseok Lee, Chandan Reddy. Local Topic Discovery via Boosted Ensemble of Nonnegative Matrix Factorization, Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI), Sister Conferences track, 2017. [pdf]
    37. Sangho Suh, Jaegul Choo, Joonseok Lee, Chandan Reddy. Boosted L-EnsNMF: Local Topic Discovery via Ensemble of Nonnegative Matrix Factorization, Proceedings of the IEEE International Conference on Data Mining (ICDM), 2016. Best Student Paper Award [pdf]
    38. Joonseok Lee, Ariel Fuxman, Bo Zhao, Yuanhua Lv. Leveraging Knowledge Bases for Contextual Entity Exploration, Proceedings of the 21st ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), 2015. [pdf] [talk]
    39. Joonseok Lee, Kisung Lee, Jennifer G. Kim, Sookyung Kim. Personalized Academic Research Paper Recommendation System, Proceedings of the 6th International Workshop on Social Recommender Systems (SRS), 2015. [pdf]
    40. Seungyeon Kim, Joonseok Lee, Guy Lebanon, Haesun Park. Local Context Sparse Coding, Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI), 2015. [pdf]
    41. Seungyeon Kim, Joonseok Lee, Guy Lebanon, Haesun Park. Estimating Temporal Dynamics of Human Emotions, Proceedings of the 29th AAAI Conference on Artificial Intelligence (AAAI), 2015. [pdf]
    42. Joonseok Lee, Samy Bengio, Seungyeon Kim, Guy Lebanon, Yoram Singer. Local Collaborative Ranking, Proceedings of the 23rd International World Wide Web Conference (WWW), 2014. Best Student Paper Award [pdf]
    43. Joonseok Lee, Seungyeon Kim, Guy Lebanon, Yoram Singer. Local Low-Rank Matrix Approximation, Proceedings of the 30th International Conference on Machine Learning (ICML), 2013. [pdf]
    44. Joonseok Lee, Seungyeon Kim, Guy Lebanon, Yoram Singer. Matrix Approximation under Local Low-Rank Assumption, The Learning Workshop in International Conference on Learning Representations (ICLR), 2013. [pdf]
    45. Mingxuan Sun, Fuxin Li, Joonseok Lee, Ke Zhou, Guy Lebanon, Hongyuan Zha. Learning Multiple-Question Decision Trees for Cold-Start Recommendation, Proceedings of the 6th ACM International Conference on Web Search and Data Mining (WSDM), 2013. [pdf]
    46. Joonseok Lee, Mingxuan Sun, Seungyeon Kim, Guy Lebanon. Automatic Feature Induction for Stagewise Collaborative Filtering, Advances in Neural Information Processing Systems (NIPS) 25, 2012. [pdf]
    47. Joonseok Lee, Hanggjun Cho, Robert Ian (Bob) McKay. Rapid Screening of Keyboard Layouts, Proceedings of IEEE International Conference on Systems, Man, and Cybernetics (SMC), 2012. [pdf]
    48. Joonseok Lee, Robert Ian (Bob) McKay. Optimizing a Personalized Cellphone Keypad, Proceedings of the 5th International Conference on Convergence and Hybrid Information Technology (ICHIT), 2011. [pdf]
  • Workshops, Demos, and ArXiv Preprints
    1. Judith Li, Kun Su, Qingqing Huang, Dima Kuzmin, Joonseok Lee, Chris Donahue, Fei Sha, Aren Jansen, Yu Wang, Mauro Verzetti, Timo Denk. V2Meow: Meowing to the Visual Beat via Music Generation, Advances in Neural Information Processing Systems (NeurIPS), Workshop on Machine Learning for Creativity and Design, 2023. [pdf]
    2. Youngwoo Cho, Seunghoon Yi, Sookyung Kim, Hongkee Yoon, Joonseok Lee. Hybrid Diffusions for Stable Molecular Structure Generation via Explicit Energy-based Model, Proceedings of the International Conference on Machine Learning (ICML), Synergy of Scientific and Machine Learning Modeling Workshop, 2023. [pdf]
    3. Seo Young Lee, Seongsu Ha, Joonseok Lee. Disentangled Audio-Driven NeRF: Talking Head Generation with Detailed Identity-Specific Micro expressions, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Sight and Sound Workshop, 2023. [pdf]
    4. Judith Yue Li, Aren Jansen, Qingqing Huang, Ravi Ganti, Joonseok Lee, Dima Kuzmin. MAQA: A Multimodal QA Benchmark for Negation, Advances in Neural Information Processing Systems (NeurIPS), Workshop on Synthetic Data for Empowering ML Research (SyntheticData4ML), 2022. [pdf]
    5. Sangho Lee, Seoyoung Lee, Joonseok Lee. Towards Detailed Characteristic-Preserving Virtual Try-On, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), The 5th Workshop on Computer Vision for Fashion, Art, and Design, 2022. [pdf]
    6. Jonghwan Mun, Minchul Shin, Gunsoo Han, Sangho Lee, Seongsu Ha, Joonseok Lee, Eun-Sol Kim. Boundary-aware Self-supervised Learning for Video Scene Segmentation, ArXiv:2201.05277, 2022. [pdf] [code]
    7. Woncheol Shin, Gyubok Lee, Jiyoung Lee, Joonseok Lee, Edward Choi. Translation-equivariant Image Quantizer for Bi-directional Image-Text Generation, ArXiv:2112.00384, 2021. [pdf]
    8. Youngwoo Cho, Hongkee Yoon, Seunghoon Yi, Jaegul Choo, Myung Joon Han, Joonseok Lee, Sookyung Kim. Deep-DFT: A Physics-ML Hybrid Approach to Predict Molecular Energy using Transformer, Advances in Neural Information Processing Systems (NeurIPS), The 4th Workshop on Machine Learning and the Physical Sciences, 2021. [pdf]
    9. Bowen Zhang, Hexiang Hu, Joonseok Lee, Ming Zhao, Sheide Chammas, Vihan Jain, Eugene Ie, Fei Sha. A Hierarchical Multi-Modal Encoder for Moment Localization in Video Corpus, ArXiv:2011.09046, 2020. [pdf]
    10. Sunghyun Park, Kangyeol Kim, Sookyung Kim, Joonseok Lee, Junsoo Le, Jiwoo Lee, Jaegul Choo. Hurricane Nowcasting with Irregular Time-step using Neural-ODE and Video Prediction, International Conference on Learning Representations (ICLR), Climate Change AI workshop, 2020. [pdf]
    11. Sookyung Kim, Sunghyun Park, Sunghyo Chung, Yunsung Lee, Hyojin Kim, Joonseok Lee, Jaegul Choo, Mr Prabhat. Focus and Track: Pixel-wise Spatio-temporal Hurricane Tracking, International Conference on Machine Learning (ICML), Workshop on Climate Change AI, 2019. [pdf]
    12. Sami Abu-El-Haija, Amol Kapoor, Bryan Perozzi, Joonseok Lee. N-GCN: Multi-scale Graph Convolution for Semi-supervised Node Classification, Proceedings of the 14th International Workshop on Mining and Learning with Graphs (MLG), 2018. [pdf]
    13. Joonseok Lee, Nisarg Kothari, Paul Natsev. Content-based Related Video Recommendations, Advances in Neural Information Processing Systems (NIPS) Demonstration Track, 2016. [pdf]
    14. Sami Abu-El-Haija, Nisarg Kothari, Joonseok Lee, Paul Natsev, George Toderici, Balakrishnan Varadarajan, Sudheendra Vijayanarasimhan. YouTube-8M: A Large-Scale Video Classification Benchmark, ArXiv:1609.08675, 2016. [pdf] [blog post]
    15. Joonseok Lee, Mingxuan Sun, Guy Lebanon. A Comparative Study of Collaborative Filtering Algorithms, ArXiv:1205.3193, 2012. [pdf]
    16. Kiyeon Lee, Joonseok Lee, Hyunwoong Shin, Sunghwan Kim, Jungwoo Choi. Real-Time Motion Tracking System using Omni-Directional PTZ Camera, Small and Medium Business Administration of South Korea, 2006. [pdf]
  • Patents
    1. Qingqing Huang, Wei Han, Daniel Sung-Joon Park, Aren Jansen, Yue Li, Ravi Ganti, Joonseok Lee, Dan Ellis, Tao Wang, Timo Immanuel Denk. Diffusion Models for Generation of Audio Data based on Semantically Descriptive Textual Prompts, US Patent (Application US63/481,746), 2023/1/26.
    2. Sangho Lee, Seoyoung Lee, Joonseok Lee. A Method and Apparatus for Trying on a Virtual Costume that Supports Various Poses and Body Types and Accurately Refleccts the Detailed Characteristics of the Costume, Korea Patent (Application 10-2022-0116656), 2022/9/15.
    3. Vihan Jain, Joonseok Lee, Ming Zhao, Sheide Chammas, Hexiang Hu, Bowen Zhang, Fei Sha, Way Tze, Eugene Ie. Hierarchical Video Encoders, US Patent US20220256175A1 (Application US17/162,150), 2022/8/11.
    4. Joonseok Lee, Balakrishnan Varadarajan, Ariel Gordon, Apostol Ivanov Natsev, Seong Jae Hwang. Framework for training machine-learned models on extremely large datasets, US Patent US11295171B2 (Application US16/657,042), 2021/4/22.
  • Theses
    1. Joonseok Lee. Local Approaches for Collaborative Filtering, Georgia Institute of Technology (Ph.D. Dissertation), 2015. [pdf]
    2. Joonseok Lee. Merging Algorithm for Unified Communicator, Seoul National University (Bachelor's Thesis), 2009. [pdf]

Talks and Presentations


Academic Activities


Ongoing Projects

  • Visual-language multi-modal representation learning
  • Video generation using ODE
  • Session-based recommendations

Past Projects

  • Video contents analysis and recommendation
  • Local Collaborative Ranking
  • Local Low-Rank Matrix Approximation
  • Local Context Sparse Coding
  • Graph-based Context-aware Entity Recommendation
  • Ensembles of collaborative filtering algorithms
  • Comparative study of collaborative filtering algorithms
    • Full experimental results (to be added soon)
  • PREA (Personalized Recommendation Algorithms) Toolkit
  • Mood analysis for blog texts

Frequent Collaborators


Interns & Students

  • Ph.D. Genealogy