profile

Sanghyuk Chun

I'm a lead research scientist at NAVER AI Lab, working on machine learning and its applications. In particular, my research interests focus on bridging the gap between two gigantic topics: reliable machine learning tasks (e.g., robustness [C3, C9, C10, W1, W3, A5], de-biasing or domain generalization [C6, C15, A5, C16], algorithmic fairness [A6], uncertainty estimation [C11, A3], explainability [C5, C11, A2, A4, W2], and fair evaluation [C5, C11]) and learning with large-scale extra data but limited annotations (e.g., multi-modal learning [C11], weakly-supervised learning [C2, C3, C4, C5, C7, C8, C12, W2, W4, W5, W6, A2, A4], and self-supervised learning). I have contributed large-scale machine learning algorithms [C3, C9, C10, C13, WO1, C17] in NAVER AI Lab as well. Prior to working at NAVER, I worked as a research engineer at the advanced recommendation team in Kakao from 2016 to 2018.

I received a master's degree in Electrical Engineering from Korea Advanced Institute of Science and Technology (KAIST) in 2016. During the master's degree, I researched on a scalable algorithm for robust subspace clustering (the algorithm is based on robust PCA and k-means clustering). Before my master's study, I worked at IUM-SOCIUS in 2012 as a software engineering internship. I also did a research internship at Networked and Distributed Computing System Lab in KAIST and NAVER Labs during summer 2013 and fall 2015, respectively.

NAVER AI Lab is looking for motivated research internship students / regular research scientists (topic: not limited! Before you apply to NAVER AI Lab, please read carefully our job decription first. If you are interested in collaborating with me, my focused research topics are: real-world biases, uncertainty estimation, robustness, causality, explainability, algorithmic fairness, multi-modal learning, vision-and-language, ...). Our mission is to perform impactful long-term AI research to make AI more beneficial and contribute to the AI community. We, therefore, expect very strong publication records (e.g., 2+ top-tier conference papers to regular research scientists, 1+ research papers to interns) for the applicants. If you are interested in joining our group, please send an email to me (or naverai at navercorp.com) with your academic CV and desired topics.

If you are interested in the internship position, you have to aware that we expect 6-month internship, and no extension is available due to legal regulations. Therefore, we expect internship students to finish their research project during 6-months (i.e., submitting a full paper to top-tier conferences, releasing their code officially, ...). It is really tough to us as well, so we'd like to keep the number of interns as small as possible (in my case, no more than three). Therefore, as of now, we are not hiring undergraduate students (or students who don't have enough publication records), and we wouldn't hire internship students because we already have our full hands. Officially, all of us work remotely untill March 2022 (no detailed plan after March as of now, but our office is located at Seoul, Korea [Google map]). Lastly, our hiring process is notoriously slow (sorry, but this is out of my hands), usually taking more than 2 months. So, please do not contact to us imminently.

News

  • _1/2022 : 2 papers [ViDT] [WCST-ML] are accepted at ICLR 2022.
  • 12/2021 : Co-hosting NeurIPS'21 workshop on ImageNet: Past, Present, and Future with 400+ attendees!
  • 12/2021 : Giving a talk at University of Seoul (topic: "Realistic challenges and limitations of AI") [slide]
  • 11/2021 : Giving a talk at NAVER and NAVER Labs Europe (topic: Mitigating dataset biases in Real-world ML applications) [slide]
  • 11/2021 : Giving a talk at UNIST (topic: Limits and Challenges in Deep Learning Optimizers) [slide]
  • 10/2021 : Releasing an unified few-shot font generation framework! [code]
  • _9/2021 : 2 papers [SWAD] [NHA] are accepted at NeurIPS 2021.
  • _8/2021: Reaching a research milestone of 1,000 citations at Google Scholar and Semantic Scholar!
  • _7/2021 : Co-organizing the NeurIPS Workshop on ImageNet: Past, Present, and Future! [webpage]
  • _7/2021 : 2 papers [MX-Font] [PiT] are accepted at ICCV 2021.
  • _7/2021 : Giving a talk at Computer Vision Centre (CVC), UAB (topic: PCME and AdamP) [info] [slide]
  • _6/2021 : Giving a talk at KSIAM 2021 (topic: AdamP). [slide]
  • _6/2021 : Giving a talk at Seoul National University (topic: few-shot font generation) .[slide]
  • _5/2021 : Receiving an outstanding reviewer award at CVPR 2021.
  • _4/2021 : 1 paper [LF-Font] is accepted at CVPR 2021 workshop (also appeared at AAAI).
  • _3/2021 : 2 papers [PCME] [ReLabel] are accepted at CVPR 2021.
  • _1/2021 : 1 paper [AdamP] is accepted at ICLR 2021.
See older news
  • _7/2020 : 1 paper [DM-Font] is accepted at ECCV 2020.
  • _6/2020 : Receiving the best paper runner-up award at AICCW CVPR 2020.
  • _6/2020 : Receiving an outstanding reviewer award at CVPR 2020.
  • _6/2020 : Giving a talk at CVPR 2020 NAVER interative session.
  • _6/2020 : 1 paper [ReBias] is accepted at ICML 2020.
  • _4/2020 : 1 paper [DM-Font short] is accepted at CVPR 2020 workshop.
  • _2/2020 : 1 paper [wsoleval] is accepted at CVPR 2020.
  • _1/2020 : 1 paper [HCNN] is accepted at ICASSP 2020.
  • 10/2019 : 1 paper [HCNN short] is accpeted at ISMIR late break demo.
  • 10/2019 : Working at Naver Labs Europe as a visiting researcher (Oct - Dec 2019)
  • _7/2019 : 2 papers [CutMix] [WCT2] are accepted at ICCV 2019 (1 oral presentation).
  • _6/2019 : Giving a talk at ICML 2019 Expo workshop.
  • _5/2019 : 2 papers [MTSA] [RegEval] are accepted at ICML 2019 workshops (1 oral presentation).
  • _5/2019 : Giving a talk at ICLR 2019 Expo talk.
  • _3/2019 : 1 paper [PRM] is accepted at ICLR 2019 workshop.

    • Publications

    (C: peer-reviewed conference, W: peer-reviewed workshop, A: arxiv preprint, O: others)
    (❋authors contributed equally)
    See also at my Google Scholar.

    Selected Publications
    • Probabilistic Embeddings for Cross-Modal Retrieval.
    • AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights.
    • Learning De-biased Representations with Biased Representations.
    • SWAD: Domain Generalization by Seeking Flat Minima.
      • Junbum Cha, Sanghyuk Chun❋, Kyungjae Lee❋, Han-Cheol Cho, Seunghyun Park, Yunsung Lee, Sungrae Park
      • NeurIPS 2021. paper | code | bibtex
    • CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features.
    • An Empirical Evaluation on Robustness and Uncertainty of Regularization methods.
      • Sanghyuk Chun, Seong Joon Oh, Sangdoo Yun, Dongyoon Han, Junsuk Choe, Youngjoon Yoo
      • ICML Workshop 2019. paper | bibtex
    • Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels.
      • Sangdoo Yun, Seong Joon Oh, Byeongho Heo, Dongyoon Han, Junsuk Choe, Sanghyuk Chun
      • CVPR 2021. paper | code | bibtex
    • Rethinking Spatial Dimensions of Vision Transformers.
      • Byeongho Heo, Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Junsuk Choe, Seong Joon Oh
      • ICCV 2021. paper | code | tweet | bibtex
    • Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Experts.
      • Song Park, Sanghyuk Chun, Junbum Cha, Bado Lee, Hyunjung Shim
      • ICCV 2021. paper | code | bibtex
    2022
    • ViDT: An Efficient and Effective Fully Transformer-based Object Detector.
      • Hwanjun Song, Deqing Sun, Sanghyuk Chun, Varun Jampani, Dongyoon Han, Byeongho Heo, Wonjae Kim, Ming-Hsuan Yang
      • ICLR 2022. paper | code | bibtex
    • Which Shortcut Cues Will DNNs Choose? A Study from the Parameter-Space Perspective.
      • Luca Scimeca❋, Seong Joon Oh❋, Sanghyuk Chun, Michael Poli, Sangdoo Yun
      • ICLR 2022. paper | bibtex
    2021
    • Few-shot Font Generation with Weakly Supervised Localized Representations.
      • Song Park❋, Sanghyuk Chun❋, Junbum Cha, Bado Lee, Hyunjung Shim
      • Submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI). (Under major revision)
      • preprint. paper | code (old) | code (new) | project page | bibtex
    • Learning Fair Classifiers with Partially Annotated Group Labels.
      • Sangwon Jung, Sanghyuk Chun, Taesup Moon
      • preprint. paper | bibtex
    • SWAD: Domain Generalization by Seeking Flat Minima.
      • Junbum Cha, Sanghyuk Chun❋, Kyungjae Lee❋, Han-Cheol Cho, Seunghyun Park, Yunsung Lee, Sungrae Park
      • NeurIPS 2021. paper | code | bibtex
    • Neural Hybrid Automata: Learning Dynamics with Multiple Modes and Stochastic Transitions.
      • Michael Poli❋, Stefano Massaroli❋, Luca Scimeca, Seong Joon Oh, Sanghyuk Chun, Atsushi Yamashita, Hajime Asama, Jinkyoo Park, Animesh Garg
      • NeurIPS 2021. paper | bibtex
    • StyleAugment: Learning Texture De-biased Representations by Style Augmentation without Pre-defined Textures.
    • Rethinking Spatial Dimensions of Vision Transformers.
      • Byeongho Heo, Sangdoo Yun, Dongyoon Han, Sanghyuk Chun, Junsuk Choe, Seong Joon Oh
      • ICCV 2021. paper | code | tweet | bibtex
    • Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Experts.
      • Song Park, Sanghyuk Chun, Junbum Cha, Bado Lee, Hyunjung Shim
      • ICCV 2021. paper | code | bibtex
    • Probabilistic Embeddings for Cross-Modal Retrieval.
    • Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels.
      • Sangdoo Yun, Seong Joon Oh, Byeongho Heo, Dongyoon Han, Junsuk Choe, Sanghyuk Chun
      • CVPR 2021. paper | code | bibtex
    • AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights.
    • Few-shot Font Generation with Localized Style Representations and Factorization.
      • Song Park❋, Sanghyuk Chun❋, Junbum Cha, Bado Lee, Hyunjung Shim
      • AAAI 2021. CVPR Workshop 2021. paper | code | project page | bibtex
    2020
    • Evaluation for Weakly Supervised Object Localization: Protocol, Metrics, and Datasets.
      • Junsuk Choe❋, Seong Joon Oh❋, Sanghyuk Chun, Seungho Lee, Zeynep Akata, Hyunjung Shim
      • Submitted to IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI). (Under minor revision)
      • preprint. paper | code and dataset | bibtex
    • Few-shot Compositional Font Generation with Dual Memory.
      • Junbum Cha, Sanghyuk Chun, Gayoung Lee, Bado Lee, Seonghyeon Kim, Hwalsuk Lee
      • ECCV 2020. paper | code | video | bibtex
    • Learning De-biased Representations with Biased Representations.
    • Toward High-quality Few-shot Font Generation with Dual Memory. Oral presentation The best paper runner-up award
      • Junbum Cha, Sanghyuk Chun, Gayoung Lee, Bado Lee, Seonghyeon Kim, Hwalsuk Lee
      • CVPR Workshop 2020. paper | bibtex
    • Evaluating Weakly Supervised Object Localization Methods Right.
    • Data-driven Harmonic Filters for Audio Representation Learning.
    2019
    • Neural Approximation of Auto-Regressive Process through Confidence Guided Sampling.
      • YoungJoon Yoo, Sanghyuk Chun, Sangdoo Yun, Jung-Woo Ha, Jaejun Yoo
      • preprint. paper | bibtex
    • Toward Interpretable Music Tagging with Self-attention.
    • CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features. Oral presentation
    • Photorealistic Style Transfer via Wavelet Transforms.
    • Automatic Music Tagging with Harmonic CNN.
      • Minz Won, Sanghyuk Chun, Oriol Nieto, Xavier Serra
      • ISMIR LBD 2019. paper | code | bibtex
    • An Empirical Evaluation on Robustness and Uncertainty of Regularization methods.
      • Sanghyuk Chun, Seong Joon Oh, Sangdoo Yun, Dongyoon Han, Junsuk Choe, Youngjoon Yoo
      • ICML Workshop 2019. paper | bibtex
    • Visualizing and Understanding Self-attention based Music Tagging. Oral presentation
    • Where To Be Adversarial Perturbations Added? Investigating and Manipulating Pixel Robustness Using Input Gradients.
      • Jisung Hwang❋, Younghoon Kim❋, Sanghyuk Chun❋, Jaejun Yoo, Ji-Hoon Kim, Dongyoon Han
      • ICLR Workshop 2019. paper | bibtex
    ~ 2018
    • Multi-Domain Processing via Hybrid Denoising Networks for Speech Enhancement.
    • A Study on Intelligent Personalized Push Notification with User History.
      • Hyunjong Lee, Youngin Jo, Sanghyuk Chun, Kwangseob Kim
      • Big Data 2017. paper | bibtex
    • Scalable Iterative Algorithm for Robust Subspace Clustering: Convergence and Initialization.
      • Master's Thesis, Korea Advanced Institute of Science and Technology, 2016 (advised by Jinwoo Shin) paper | code

    Academic Activities

    Professional Service
    • Reviewer:
      • ICML 2021-2022, NeurIPS 2020-2021, ICLR 2021-2022, AAAI 2021, CVPR 2020-2022, ICCV 2021, WACV 2021, ACCV 2020
    • Outstanding reviewer:
      • CVPR 2020, CVPR 2021
    • NeurIPS 2021 Workshop on ImageNet: Past, Present, and Future
      • Co-organized by Zeynep Akata, Lucas Beyer, Sanghyuk Chun, Almut Sophia Koepke, Diane Larlus, Seong Joon Oh, Rafael Sampaio de Rezende, Sangdoo Yun, Xiaohua Zhai
    Awards
    • Outstanding reviewer award, CVPR 2021
    • Outstanding reviewer award, CVPR 2020
    • Best paper runner-up award, AI for Content Creation Workshop at CVPR 2020
    Talks
    • "Realistic challenges and limitations of AI", University of Seoul. [slide]
    • "Mitigating dataset biases in Real-world ML applications", NAVER and NAVER Labs Europe (2021). [slide]
    • "Limits and Challenges in Deep Learning Optimizers", UNIST (2021). [slide]
    • "Towards better cross-modal learning by Probabilistic embedding and AdamP optimizer", UAB CVC (2021). [info] [slide]
    • "AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights", KSIAM (2021). [slide]
    • "Towards Few-shot Font Generation", Seoul University and NAVER (2021). [slide]
    • "Learning De-biased Representations with Biased Representations", NAVER (2020). [slide]
    • "Reliable Machine Learning in NAVER AI", Yonsei University (2020). [slide]
    • "Toward Reliable Machine Learning", omnious and nota (2020). [slide]
    • "Reliable Machine Learning", NAVER CVPR 2020 sponser event. [program] [slide] [video]
    • "Neural Architectures for Music Representation Learning", NAVER (2020). [slide]
    • "Learning generalizable representations with CutMix and ReBias", NAVER Labs Europe (2019).
    • "An empirical evaluation on the generalizability of regularization methods", ICML 2019 Expo Talk: Recent Work on Machine Learning at NAVER. [slide]
    • "Recent works on deep learning robustness in Clova AI", ICLR 2019 Expo Talk: Representation Learning to Rich AI Services in NAVER and LINE.
    • "Recommendation system in the real world", Deepest Summer School 2018. [slide]

    Mentoring and Teaching

    Mentees / Short-term post-doctoral collaborators / Internship students

    Topics: Reliable ML learning with limited annotations Modality-specific tasks Generative models Other topics

    • _    Saehyung Lee (Seoul National University, 2021-2022) under review papers -- Adversarial robustness
    • _ _ Sangwon Jung (Seoul National University, 2021-2022) [A8] -- Fairness
    • _    Luca Scimeca (A short-term post-doctoral collaborator, 2021) [A6] [C14] -- Understanding shortcut learning phenomenon in feature space
    • _    Michael Poli (KAIST, 2021) [C14] [A6] -- Neural hybrid automata
    • _    Hyemi Kim (KAIST, 2021) -- Test-time training for robust prediction
    • _    Jun Seo (KAIST, 2021) -- Self-supervised learning
    • _    Song Park (Yonsei University, 2020-2021) [C8/W6] [C12] [A5] [A9] -- Few-shot font generation
    • _    Hyojin Bahng (Korea University, 2019) [C6] -- De-biasing
    • _ _ Junsuk Choe (Yonsei University, 2019) [C5] [A4] -- Reliable evaluation for WSOL
    • _    Naman Goyal (IIT RPR, 2019) -- Robust representation against shift
    • _    Minz Won (Music Technology Group, Universitat Pompeu Fabra, 2018-2019) [W2] [W4] [A2] [C4] -- Audio representation learning
    • _    Byungkyu Kang (Yonsei University, 2018) [C2] -- Image-to-image translation and style transfer
    • _    Jang-Hyun Kim (Seoul National University, 2018) [A1] -- Audio representation learning
    • _    Jisung Hwang (University of Chicago, 2018) [W1] -- Adversarial robustness
    • _    Younghoon Kim (Seoul National University, 2018) [W1] -- Adversarial robustness
    Guest lectures
    • "Limits and Challenges in Deep Learning Optimizers", UNIST (2021). [slide]
    • "Towards Few-shot Font Generation", Seoul University and NAVER (2021). [slide]
    • "Reliable Machine Learning in NAVER AI", Yonsei University (2020). [slide]
    • "Recommendation system in the real world", Deepest Summer School 2018. [slide]

    Industry Experience

    NAVER AI Research (2018 ~ Now)
    • Hangul
      Hangul
      DM-Font teasor
      Hangul Handwriting Font Generation

      Distributed at 2019 Hangul's day (한글날), [Full font list]

      • Hangul (Korean alphabet, 한글) originally consists of only 24 sub-letters (ㄱ, ㅋ, ㄴ, ㄷ, ㅌ, ㅁ, ㅂ, ㅍ, ㄹ, ㅅ, ㅈ, ㅊ, ㅇ, ㅎ, ㅡ, ㅣ, ㅗ, ㅏ, ㅜ, ㅓ, ㅛ, ㅑ, ㅠ, ㅕ), but by combining them, there exist 11,172 valid characters in Hangul. For example, "한" is a combination of ㅎ, ㅏ, and ㄴ, and "쐰" is a combination of ㅅ, ㅅ, ㅗ, ㅣ, and ㄴ. It makes generating a new Hangul font be very expensive and time-consuming. Meanwhile, since 2008, Naver has distributed Korean fonts for free (named Nanum fonts, 나눔 글꼴).
      • In 2019, we developed a technology for fully-personalized Hangul generation only with 152 characters. We opened an event page where users can submit their own handwriting. The full generated font list can be found in [this link]. Details for the generation technique used for the service was presented in Deview 2019 [Link].
      • This work was also extended to the few-shot generation based on the compositionality. See the papers in AI for Content Creation Workshop (AICCW) at CVPR 2020 (short paper) [Link], ECCV 2020 (full paper) [Link], AAAI 2021 [Link], and ArXiv preprint [Link].
      • [BONUS] You can play with my handwriting here
    • example sticker
      Example emoji from LINE sticker shop.
      Emoji Recommendation (LINE Timeline)

      Deployed in Jan. 2019

      • LINE is a major messenger player in east asia (Japan, Taiwan, Thailand, Indonesia, and Korea). In the application, users can buy and use numerous emoijs a.k.a. LINE Sticker.
      • In this project, we recommended emojis to users based on their profile picture (cross-domain recommendation).
      • I developed and researched the entire pipeline of the cross-domain recommendation system and operation tools.
    Kakao Advanced Recommendation Technology (ART) team (2016 ~ 2018)
    • Kakao
      Recommender Systems (Kakao services)

      Feb. 2016 - Feb. 2018

      • I developed and maintained a large-scale real-time recommender system (Toros [PyCon Talk] [AI Report]) for various services in Daum and Kakao. I mainly worked with content-based representation modeling (for textual, visual, and musical data), collaborative filtering modeling, user embedding, user clustering, and ranking system based on Multi-armed Bandit.
      • Textual domain: Daum News similar article recommendation, Brunch (blog service) similar post recommendation, Daum Cafe (community service) hit item recommendation.
      • Visual domain: Daum Webtoon and Kakao Page similar item recommendation, video recommendation for a news article (cross-domain recommendation).
      • Audio domain: music recommendation for Kakao Mini (smart speaker), Melon and Kakao Music.
      • Online to offline: Kakao Hairshop style recommendation.
    • IPPN
      System overview.
      Personalized Push Notification with User History (Daum, Kakao Page)

      Deployed in 2017

      • The mobile push service (or alert system) is widely-used in mobile applications to attain a high user retention rate. However, a freqeunt push notification makes a user feel fatigue, resulting on the application removal. Usually, the push notification system is a rule-based system, and managed by human labor. In this project, we researched and developed a personalized push notification system based on user activity and interests. The system has been applied to Daum an Kakao Page mobile applications. More details are in our paper.
    • Daum Shopping
      Large-Scale Item Categorization in e-Commerce (Daum Shopping)

      Deployed in 2017

      • An accurate categorization helps users to search desired items in e-Commerce based on the category, e.g., clothes / shoes / sneakers. However, the categorization is usually performed based on rule-based systems or human labor, which leads to low coverage of categorized items. Even the automatic item categorization is difficult due to its web-scale data size, the highly unbalanced annotation distribution, and noisy labels. I developed a large-scale item categorization system for Daum Shopping based on a deep network, from the operation tool to the categorization API.
    Internship
    • Naver Labs
      Research internship (Naver Labs)

      Aug. 2015 - Dec. 2015

      • During the internship, I implemented batch normalization (BN) to AlexNet, Inception v2 and VGG on ImageNet using Caffe. I also researched batch normalization for sequential models, e.g., RNN using Lua Torch.
    • IUM-SOCIUS
      Software engineer (IUM-SOCIUS)

      Jun. 2012 - Jan. 2013

      • I worked as web developer at IUM-SOCIUS. During the internship, I developed and maintained internal batch services (JAVA spring batch), internal statistics service (Python Flask, MongoDB), internal admin tools (Python Django, MySQL), and main service systems (JAVA spring, Ruby on Rails, MariaDB).

    Education and Career

    • M.S. (2014.03 - 2016.02), School of Electrical Engineering, KAIST
    • B.S. (2009.03 - 2014.02), School of Electrical Engineering and School of Management Science (double major), KAIST