I'm a research scientist at NAVER AI LAB and NAVER CLOVA, working on machine learning and its applications. Prior to working at NAVER, I worked as a research engineer at advanced recommendation team (ART) in Kakao from 2016 to 2018.
I received a master's degree in Electrical Engineering from Korea Advanced Institute of Science and Technology (KAIST) in 2016. During the master's degree, I researched on a scalable algorithm for robust subspace clustering (the algorithm is based on robust PCA and k-means clustering). Before my master's study, I worked at IUM-SOCIUS in 2012 as a software engineering internship. I also did a research internship at Networked and Distributed Computing System Lab in KAIST and NAVER Labs during summer 2013 and fall 2015, respectively.
NAVER AI LAB is looking for motivated research internship students / regular research scientists (topic: real-world biases, uncertainty estimation, robustness, causality, explainability, large-scale learning, self-supervised learning, multi-modal learning). If you are interested in joining our group, please send an email to me with your academic CV and desired topics.
[For the internship] We recommend a 6-month internship (especially if you are not a Korean citizen, at least 6-month working is required for the VISA process). The location will be Seoul, Korea [Google map], but a remote internship program can be considered depending on the situation. We are not hiring an undergraduate student for the internship. Lastly, if you are not a Korean citizen, the whole hiring process could be delayed due to the VISA process.
Research Interests
Reliable machine learning with limited supervision. Real-world machine learning models often suffer from unreliability issues; (1) the lack of generalizability to unseen biases or corruptions, (2) improper uncertainty estimation, (3) their decisions are not explainable to humans. To achieve reliable machine decisions, we need a large number of annotations in every possible situation, e.g., traffic signs with every possible weather condition, which is highly impractical and unachievable in most cases. Instead of collecting or generating all possible situations, my research interests focus on developing reliable machine learning models with only limited human supervision. In particular, I am interested in the following types of supervision: (1) human inductive bias without additional labeling, (2) extra multi-modal information related to the original task, (3) weak supervision, or semi-supervision which requires a reasonable number of additional annotations.
News
- _3/2021 : 2 papers [PCME] [ReLabel] are accepted at CVPR 2021.
- _1/2021 : 1 paper [AdamP] is accepted at ICLR 2021.
- 12/2020 : 1 paper [LF-Font] is accepted at AAAI 2021.
- _7/2020 : 1 paper [DM-Font] is accepted at ECCV 2020.
- _6/2020 : Receiving the best paper runner-up award at AICCW CVPR 2020.
- _6/2020 : Receiving an outstanding reviewer award at CVPR 2020.
- _6/2020 : Giving a talk at CVPR 2020 NAVER interative session.
- _6/2020 : 1 paper [ReBias] is accepted at ICML 2020.
- _4/2020 : 1 paper [DM-Font short] is accepted at CVPR 2020 workshop.
- _2/2020 : 1 paper [wsoleval] is accepted at CVPR 2020.
- _1/2020 : 1 paper [HCNN] is accepted at ICASSP 2020.
- 10/2019 : 1 paper [HCNN short] is accpeted at ISMIR late break demo.
- 10/2019 : Working at Naver Labs Europe as a visiting researcher (Oct - Dec 2019)
- _7/2019 : 2 papers [CutMix] [WCT2] are accepted at ICCV 2019 (1 oral presentation).
- _6/2019 : Giving a talk at ICML 2019 Expo workshop.
- _5/2019 : 2 papers [MTSA] [RegEval] are accepted at ICML 2019 workshops (1 oral presentation).
- _5/2019 : Giving a talk at ICLR 2019 Expo talk.
- _3/2019 : 1 paper [PRM] is accepted at ICLR 2019 workshop.
Publications
(C: peer-reviewed conference, W: peer-reviewed workshop, A: arxiv preprint, O: others)
(❋authors contributed equally)
See also at my Google Scholar.
Selected Publications
- Probabilistic Embeddings for Cross-Modal Retrieval.
-
AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights.
- Byeongho Heo❋, Sanghyuk Chun❋, Seong Joon Oh, Dongyoon Han, Sangdoo Yun, Gyuwan Kim, Youngjung Uh, Jung-Woo Ha
- ICLR 2021. paper | code | project page | pypi | bibtex
- Learning De-biased Representations with Biased Representations.
-
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features.
- Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, Youngjoon Yoo
- ICCV 2019. paper | code and pretrained models | blog | bibtex
2021
- Rethinking Spatial Dimensions of Vision Transformers.
- Multiple Heads are Better than One: Few-shot Font Generation with Multiple Localized Experts.
- Probabilistic Embeddings for Cross-Modal Retrieval.
- Re-labeling ImageNet: from Single to Multi-Labels, from Global to Localized Labels.
-
AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights.
- Byeongho Heo❋, Sanghyuk Chun❋, Seong Joon Oh, Dongyoon Han, Sangdoo Yun, Gyuwan Kim, Youngjung Uh, Jung-Woo Ha
- ICLR 2021. paper | code | project page | pypi | bibtex
- Few-shot Font Generation with Localized Style Representations and Factorization.
2020
-
Evaluation for Weakly Supervised Object Localization: Protocol, Metrics, and Datasets.
- Junsuk Choe❋, Seong Joon Oh❋, Sanghyuk Chun, Zeynep Akata, Hyunjung Shim
- preprint. paper | code and dataset | bibtex
- Few-shot Compositional Font Generation with Dual Memory.
- Learning De-biased Representations with Biased Representations.
- Toward High-quality Few-shot Font Generation with Dual Memory. Oral presentation The best paper runner-up award
-
Evaluating Weakly Supervised Object Localization Methods Right.
- Junsuk Choe❋, Seong Joon Oh❋, Seongho Lee, Sanghyuk Chun, Zeynep Akata, Hyunjung Shim
- CVPR 2020. paper | code and dataset | tweet | slide | video on CVPR | video on ECCV tutorial | bibtex
-
Data-driven Harmonic Filters for Audio Representation Learning.
- Minz Won, Sanghyuk Chun, Oriol Nieto, Xavier Serra
- ICASSP 2020. paper | code and pretrained models | video | bibtex
2019
- Neural Approximation of Auto-Regressive Process through Confidence Guided Sampling.
-
Toward Interpretable Music Tagging with Self-attention.
- Minz Won, Sanghyuk Chun, Xavier Serra
- preprint. paper | code and pretrained models | bibtex
-
CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features. Oral presentation
- Sangdoo Yun, Dongyoon Han, Seong Joon Oh, Sanghyuk Chun, Junsuk Choe, Youngjoon Yoo
- ICCV 2019. paper | code and pretrained models | blog | bibtex
-
Photorealistic Style Transfer via Wavelet Transforms.
- Jaejun Yoo❋, Youngjung Uh❋, Sanghyuk Chun❋, Byungkyu Kang, Jung-Woo Ha
- ICCV 2019. paper | code and model weights | video | blog | bibtex
- Automatic Music Tagging with Harmonic CNN.
- An Empirical Evaluation on Robustness and Uncertainty of Regularization methods.
-
Visualizing and Understanding Self-attention based Music Tagging. Oral presentation
- Minz Won, Sanghyuk Chun, Xavier Serra
- ICML Workshop 2019. paper | code | talk video | bibtex
- Where To Be Adversarial Perturbations Added? Investigating and Manipulating Pixel Robustness Using Input Gradients.
~ 2018
-
Multi-Domain Processing via Hybrid Denoising Networks for Speech Enhancement.
- Jang-Hyun Kim❋, Jaejun Yoo❋, Sanghyuk Chun, Adrian Kim, Jung-Woo Ha
- preprint. paper | project page | bibtex
- A Study on Intelligent Personalized Push Notification with User History.
-
Scalable Iterative Algorithm for Robust Subspace Clustering: Convergence and Initialization.
- Master's Thesis, Korea Advanced Institute of Science and Technology, 2016 (advised by Jinwoo Shin) paper | code
Academic Activities
Professional Service
- Reviewer:
- ICCV 2021
- ICML 2021
- NeurIPS 2020
- ICLR 2021
- AAAI 2021
- CVPR 2020 (outstanding reviewer), 2021
- ICCV 2021
- WACV 2021
- ACCV 2020
Awards
- Outstanding reviewer award, CVPR 2020
- Best paper runner-up award, AI for Content Creation Workshop at CVPR 2020
Talks
- "Reliable Machine Learning in NAVER AI", Yonsei University (2020). [slide]
- "Toward Reliable Machine Learning", omnious and nota (2020). [slide]
- "Reliable Machine Learning", NAVER CVPR 2020 sponser event. [program] [slide] [video]
- "Neural Architectures for Music Representation Learning", NAVER (2020). [slide]
- "Learning generalizable representations with CutMix and ReBias", NAVER Labs Europe (2019).
- "An empirical evaluation on the generalizability of regularization methods", ICML 2019 Expo Talk: Recent Work on Machine Learning at NAVER.
- "Recent works on deep learning robustness in Clova AI", ICLR 2019 Expo Talk: Representation Learning to Rich AI Services in NAVER and LINE.
- "Recommendation system in the real world", Deepest Summer School 2018.
Industry Experience
NAVER AI Research (2018 ~ Now)
-
Hangul Handwriting Font GenerationDistributed at 2019 Hangul's day (한글날), [Full font list]
- Hangul (Korean alphabet, 한글) originally consists of only 24 sub-letters (ㄱ, ㅋ, ㄴ, ㄷ, ㅌ, ㅁ, ㅂ, ㅍ, ㄹ, ㅅ, ㅈ, ㅊ, ㅇ, ㅎ, ㅡ, ㅣ, ㅗ, ㅏ, ㅜ, ㅓ, ㅛ, ㅑ, ㅠ, ㅕ), but by combining them, there exist 11,172 valid characters in Hangul. For example, "한" is a combination of ㅎ, ㅏ, and ㄴ, and "쐰" is a combination of ㅅ, ㅅ, ㅗ, ㅣ, and ㄴ. It makes generating a new Hangul font be very expensive and time-consuming. Meanwhile, since 2008, Naver has distributed Korean fonts for free (named Nanum fonts, 나눔 글꼴).
- In 2019, we developed a technology for fully-personalized Hangul generation only with 152 characters. We opened an event page where users can submit their own handwriting. The full generated font list can be found in [this link]. Details for the generation technique used for the service was presented in Deview 2019 [Link].
- This work was also extended to the few-shot generation based on the compositionality. See the papers in AI for Content Creation Workshop (AICCW) at CVPR 2020 (short paper) [Link], ECCV 2020 (full paper) [Link], and AAAI 2021 [Link].
- [BONUS] You can play with my handwriting here
-
Example emoji from LINE sticker shop. Emoji Recommendation (LINE Timeline)Deployed in Jan. 2019
- LINE is a major messenger player in east asia (Japan, Taiwan, Thailand, Indonesia, and Korea). In the application, users can buy and use numerous emoijs a.k.a. LINE Sticker.
- In this project, we recommended emojis to users based on their profile picture (cross-domain recommendation).
- I developed and researched the entire pipeline of the cross-domain recommendation system and operation tools.
Kakao Advanced Recommendation Technology (ART) team (2016 ~ 2018)
-
Recommender Systems (Kakao services)Feb. 2016 - Feb. 2018
- I developed and maintained a large-scale real-time recommender system (Toros [PyCon Talk] [AI Report]) for various services in Daum and Kakao. I mainly worked with content-based representation modeling (for textual, visual, and musical data), collaborative filtering modeling, user embedding, user clustering, and ranking system based on Multi-armed Bandit.
- Textual domain: Daum News similar article recommendation, Brunch (blog service) similar post recommendation, Daum Cafe (community service) hit item recommendation.
- Visual domain: Daum Webtoon and Kakao Page similar item recommendation, video recommendation for a news article (cross-domain recommendation).
- Audio domain: music recommendation for Kakao Mini (smart speaker), Melon and Kakao Music.
- Online to offline: Kakao Hairshop style recommendation.
-
System overview. Personalized Push Notification with User History (Daum, Kakao Page)Deployed in 2017
- The mobile push service (or alert system) is widely-used in mobile applications to attain a high user retention rate. However, a freqeunt push notification makes a user feel fatigue, resulting on the application removal. Usually, the push notification system is a rule-based system, and managed by human labor. In this project, we researched and developed a personalized push notification system based on user activity and interests. The system has been applied to Daum an Kakao Page mobile applications. More details are in our paper.
-
Large-Scale Item Categorization in e-Commerce (Daum Shopping)Deployed in 2017
- An accurate categorization helps users to search desired items in e-Commerce based on the category, e.g., clothes / shoes / sneakers. However, the categorization is usually performed based on rule-based systems or human labor, which leads to low coverage of categorized items. Even the automatic item categorization is difficult due to its web-scale data size, the highly unbalanced annotation distribution, and noisy labels. I developed a large-scale item categorization system for Daum Shopping based on a deep network, from the operation tool to the categorization API.
Internship
-
Research internship (Naver Labs)Aug. 2015 - Dec. 2015
- During the internship, I implemented batch normalization (BN) to AlexNet, Inception v2 and VGG on ImageNet using Caffe. I also researched batch normalization for sequential models, e.g., RNN using Lua Torch.
-
Software engineer (IUM-SOCIUS)Jun. 2012 - Jan. 2013
- I worked as web developer at IUM-SOCIUS. During the internship, I developed and maintained internal batch services (JAVA spring batch), internal statistics service (Python Flask, MongoDB), internal admin tools (Python Django, MySQL), and main service systems (JAVA spring, Ruby on Rails, MariaDB).
Education and Career
- M.S. (2014.03 - 2016.02), School of Electrical Engineering, KAIST
- B.S. (2009.03 - 2014.02), School of Electrical Engineering and School of Management Science (double major), KAIST
- Tech leader / research scientist at NAVER AI LAB and NAVER CLOVA (Feb. 2018 - Now)
- Visiting researcher at Naver Labs Europe (Oct. 2019 - Dec. 2019)
- Research engineer at advanced recommendation team (ART) in Kakao (Feb. 2016 - Feb. 2018)
- Research internship at Naver Labs (Aug. 2015 - Dec. 2015)
- Master's degree in Electrical Engineering from KAIST (advisor: Jinwoo Shin) (Mar. 2014 - Feb. 2016)
- Undergraduate researcher at Algorithmic Intelligence Lab in KAIST (Fall 2013)
- Undergraduate researcher at Networked and Distributed Computing System Lab in KAIST (Summer 2013)
- Participated as an undergraduate researcher for FloSIS: A Highly Scalable Network Flow Capture System for Fast Retrieval and Storage Efficiency (presented at USENIC ATC 2015) (I developed the index system described in Section 4.)
- Software engineering internship at IUM-SOCIUS (Jun. 2012 - Jan. 2013)