LG

LG AI Research held the “LG AI Talk Concert - The Best of 2021” on December 14 to share the achievements made throughout the year. It was an opportunity to look at the progress of the projects performed by LG AI Research at a glance and to explore the super-giant AI disclosed by LG AI Research. Although it was in the form of live streaming through the YouTube channel, the talk concert, which introduced new groundbreaking achievements as well as the latest research results produced by LG AI Research, was more heated than anywhere else.
 
Keynote: Looking back on LG AI Research in 2021 and unveiling a new super-giant AI
 
The LG AI Talk Concert began with a keynote speech from Kyunghoon Bae, Representative of LG AI Research. In light of the first anniversary of LG AI Research, he shared the key research results throughout the year and unveiled Korea’s largest super-giant AI for the first time.
 
Bae started off with the key research results produced by LG AI Research. LG AI Research has conducted various basic research studies and published research papers over the past year. As a result, a total of 18 papers were accepted at global top-tier conferences. LG AI Research is the only AI research organization in Korea that has achieved such outstanding results during its first year of foundation. In addition, LG AI Research has demonstrated excellent outcomes in Machine Reading Comprehension (MRC), an area for assessing a computer that reads a given document and finds the answer to a question as a human does. LG AI Research took first place in the Korean machine reading comprehension competition, last year’s KorQuAD (Korean Question Answering Dataset), and this year’s SQuAD (Stanford Question Answering Dataset), its English counterpart.
 
AI researches that were applied to the field were also shared. One of them was about using artificial intelligence to solve industrial problems in collaboration with AI entities in affiliated companies. This year, LG AI Research has solved a total of 18 problems held by many affiliated companies of the LG group across all industries, including personalized medicine, chatbots, corporate demand forecasting, and vision inspection. It will take on more than 25 projects to solve highly complicated problems next year.
 
As such, LG AI Research wanted to provide more values in the lives of customers through the top 1% AI experts at a higher level beyond outstanding research accomplishments. To that end, LG AI Research has made continuous efforts to develop the super-giant AI that was briefly announced this May and finally unveiled to the world at this time. It is LG’s super-giant AI, EXAONE.
 
EXAONE stands for “Expert AI for Everyone,” an expert AI for everyone that LG aims for with various features. EXAONE is differentiated from previous models since it is a multi-modal super-giant AI model that can process both language and visual data. With the world’s largest learning data being used, this huge amount of data learning enabled easily creating images and texts bidirectionally. In addition, EXAONE is a bilingual artificial intelligence that has learned the characteristics of both Korean and English languages at the same time. Since the initial development last June, it has completed learning of 1.3 billion, 13 billion, 39 billion, and 175 billion parameter models, and it is currently learning 300 billion parametric models. EXAONE shows remarkable performance such as obtaining the highest FID score, offering 1024x1024 sized image output, and achieving purpose conversation in the language domain as well as the highest level in the emotional classification domain.
 
Kyunghoon Bae also mentioned the use of EXAONE, the super-giant AI developed this way. He announced the plan to build an ecosystem of EXAONE in stages, such as preparing APIs, building large-scale infrastructure, and collaborating with strategic partners.
 
Fundamental Session: Various aspects of basic research 
 
One of the strengths of LG AI Research is that it actively conducts various basic research studies. Such efforts bore fruit, with the 18 papers published by LG AI Research adopted by global top-tier conferences this year. In the Fundamental Session, three papers were introduced among the papers adopted by these leading global conferences.
 
Shortest-Path Constrained Reinforcement Learning for Sparse Reward Tasks (ICML 2021) – Presenter:  Sungryull Sohn 
 
This research paper is on “Shortest-Path Constrained Reinforcement Learning,” a method of greatly increasing the efficiency of reinforcement learning when rewards are rarely given. Through various experiments, the paper proposed a new direction wherein the Shortest-Path Reinforcement Learning (SPRL) proposed in the paper can improve reinforcement learning efficiency by simplifying the problem and preserving the optimal solution; thus achieving higher performance compared to existing methods.
 
Composer AI with tap-to-pitch generator (NeurIPS 2021 workshop accepted) – Presenter: Hyeongrae Ihm 
 
This paper introduced a deep learning-based AI model that automatically generates music. Music plays an important role in many content, but its use is often very limited due to copyright issues. According to the paper, the recently emerging trend of automatic music generation can be realized using AI. The paper introduced how AI expresses music, including a Neural Network model and structure that generates music phrases (4-8 bars that make up music) and a Composer AI that combines these phrases to create a song.
 
Rethinking Deep Image Prior for Denoising (ICCV 2021) – Presenter: Yeonsik Jo
 
This research paper is on “Denoising,” which turns images with noise into clean images with an analysis of Deep Image Prior (DIP) using the concept of Effective Degrees of Freedom. The paper used the formula called Stein Unbiased Risk Estimator and presented a methodology to supplement the formula at the same time. It also proposed a loss-based learning termination method to solve the problems of the existing DIP.
 
Applied AI Session: Solving various challenges existing in the field with AI
 
LG AI Research is conducting not only basic research for the fundamental improvement of AI but also research to solve various difficulties in the field. Among the many research studies showing that AI can be used meaningfully and is an essential technology, five significant cases were introduced in the Applied AI Session.
 
Progress and Performance of Materials AI Research – Presenter: Sehui Han Squad Leader
 
The progress and performance of material development using AI by LG AI Research, which is underway, was unveiled. LG AI Research is accelerating material development by using data and artificial intelligence technology. Among them, the sectors of bio and electronic materials were introduced in the session. The bio sector aims at the development of immuno-cancer treatment tailored to individual patients. In this sector, a myriad of experimental data is obtained to provide the learning of AI models and develop an attention-based AI model that can predict the interaction between biomolecules based on the discovered promising candidates. As a result, the prediction accuracy outperformed existing models. The electronic materials sector aims at the development of light emitting materials used in OLED displays. In this sector, AI is used to search for light emitting materials, which enables searching for candidate materials in a much shorter time than before and selecting candidate materials whose good performance is expected.
 
Progress and Performance of Next-Generation Vision Inspection Research – Presenter: Byungjun Kang Squad Leader
 
In this session, the presenter shared the progress and performances of the next-generation vision inspection research using AI for vision inspection to determine defects in the appearance of products in the manufacturing process. As deep learning technologies are increasingly applied to vision inspection, inspection performance has also been improved dramatically. LG AI Research has developed Continuous Learning technology that dramatically shortens the data preparation time and maintains the accuracy of judgment for new models through Anomaly Detection, which detects the quantity only with non-defective data. It is now preparing for commercialization by obtaining meaningful test results with field data.
 
AI-Based Battery Anomaly Detection – Presenter: Yeseul Sim Squad Leader
 
A research on AI-based detection of abnormal behaviors of batteries was also introduced. Anomaly detection is a technology that can automatically identify anomaly, an abnormal pattern that does not follow a well-defined normal pattern within the data. LG AI Research presented a way of realizing anomaly detection with an AI generation model, unlike the existing methodology using traditional statistical methods.
 
AI-Based Product/Part Demand Forecasting – Presenter: Jinseok Yang Squad Leader  
 
In this session, the presenter introduced the progress of research on Demand Forecasting, a tool that predicts future demand based on past data and supports smooth decision making to generate higher profits for the company. Demand forecasting is helpful in decision making for a range of subjects throughout the entire process from production to sales. The presenter introduced a demand forecasting model developed to reduce logistics and operating costs by forecasting demands of companies that produce products directly such as LG, unlike other retail stores. The research was found separate AI-based demand forecasting that shows optimal performance for each data.
 
Progress and Performance of Machine Reading Comprehension (MRC) Research – Presenter: Kyungkoo Min Squad Leader
 
There was also a session on introducing the progress and performance of Machine Reading Comprehension (MRC) research, wherein a question asked in natural language about an arbitrary document can be answered by searching answers directly from the document. MRC technology is useful in application areas such as customer center requiring specialized knowledge or chatbot search. It is also an area where LG AI Research has achieved excellent results in Korean and English. LG AI Research has focused on technology that improves the performance of the pre-trained language model, which is the basis of MRC quality, and re-ranks the search results in order to enhance the applicability of MRC technology. LG AI Research is also developing a technology to facilitate search from content described in tabular format.
 
Super-Giant AI Session: Unveiling EXAONE to the World
 
It was EXAONE, a super-giant AI developed by LG AI Research, that attracted the most attention at the LG AI Talk Concert. As an expert AI created by learning vast amounts of human knowledge, EXAONE is a multi-model super-giant AI model that can handle both language and visual data. EXAONE is already outperforming the SOTA model in a wide range of fields. Among many features, the vision model was introduced that day.
 
EXAONE Multi-Modal  – Presenter: Sihaeng Lee Squad Leader
 
In this session, the presenter introduced the EXAONE multi-modal model of LG AI Research. It was a time to understand the process from the start of the project to the current status and the forward direction pursued by the LG AI Research. The initial focus was on fashion and product design in the early stage of the project of realizing a super-giant AI that combines language and vision, so that it could make a contribution to the entire LG Group. The project team discovered numerous possibilities of super-giant AI through countless experiments. Based on such, they could develop a super-giant AI model that could understand images to generate texts or create images from texts.
 
EXAONE's Bi-directional Generation Algorithm – Presenter: Taehoon Kim
 
In this session, the bidirectional image-text generation algorithm in EXAONE, L-Verse, was introduced. L-Verse is a model proposed to find the correlation between vision and language in latent space. This algorithm can generate detailed captions, sometimes more than the human-written ones, without additional fine-tuning beyond the existing models wherein only text to image generation is possible. As bidirectional generation is possible, it also showed excellent performance in image generation.
 
EXAONE Tuning: New Way of Lightweight Tuning – Presenter: Janghoon Han
 
The next session was on the introduction of the tuning technique of EXAONE, a super-giant language model of LG. Until now, Fine-Tuning, In-Context Learning, or Lightweight Tuning along with P-Tuning as a representative technique among them has been suggested to tune a language model to a task. LG AI Research proposes EXAONE Tuning, a new Lightweight Tuning technique that solves the problems associated with such techniques. It was the result of studying an efficient way to improve performance while learning fewer parameters.
 
EXAONE Inference Framework – Presenter: Hyunjik Jo
 
EXAONE Inference Framework, developed to service the super-giant language model, was also introduced. Inference Framework is a framework that aims to service the learned model. Specifically, EXAONE Inference Framework is the differentiated Inference Framework to service super-giant models efficiently. LG AI Research has studied many ways to drive and load parameters of up to 175 billion, and it is currently working on improvements to the framework to provide faster and more efficient service than the existing methods. LG AI Research also filed patents for super-giant models separately by strengthening the GPU memory, reduced the main memory usage to 1/8, and developed CUDA Fused Kernel to improve speed by 2.5 times.
 
“LG AI Talk Concert - The Best of 2021” was a venue for demonstrating outstanding research results and unveil the excellence of super-giant AI. It was an opportunity to see the results of intensive efforts made by LG AI Research to make a better life through AI technology. There are high expectations that LG AI Research could create a positive virtuous cycle of solving many difficult problems and challenges through its various research and achievements and by building upon such to take on more complex tasks at the next level.