Upstage's Solar 10.7B Emerges as World's Top Pre-trained LLM
2023/12/14Upstage LLM 'Solar' released, ranked first in the world on the 'Hugging Face Open LLM Leaderboard' along with the announcement.
THE WORLD'S FIRST SMALL MODEL WITH 10.7B PARAMETERS BECAME THE GLOBAL TOP GENERATING AI MODEL.
BOTH A PRE-LEARNING MODEL CAPABLE OF ADDITIONAL LEARNING AND A FINE TUNING MODEL WITH HIGH PRACTICAL USABILITY ARE RELEASED... DRIVING THE EXPANSION OF THE SLM ECOSYSTEM
Upstage enters the global generative AI market based on top technology through cooperation with global platforms such as AWS and Poe
Seoul, Dec. 14, 2023 Upstage has officially introduced its self-developed pre-trained LLM (Large Language Model) named 'Solar.' This unveiling marks the company's active participation in the global LLM competition.
Upstage announced on the 14th that Solar achieved the top position on the 'Open LLM Leaderboard' operated by Hugging Face, the world's largest machine learning platform. This accomplishment holds significant value as Solar is recognized as the world's best-performing model with fewer than 30 billion parameters (30B), meeting the standard for Small LLM (SLM).
Upstage Solar, which stands for Specialized and Optimized LLM and Applications with Reliability, followed on the success of Upstage's Hugging Face model last August, surpassing the benchmark score of GPT-3.5.
The Hugging Face Open LLM Leaderboard serves as a key indicator for open-source AI models, evaluating around 500 models globally. It ranks models based on six indicators, including reasoning and common sense ability, comprehensive language understanding ability, hallucination prevention, pronoun reference, and math-solving ability.
Solar distinguishes itself as a small-sized pre-trained model for private LLM, making it user-friendly for companies. With 10.7 billion parameters (10.7B), Solar is the world's first model of its kind. Upstage's model strikes a balance between high intelligence and compactness, scoring 74.2 points in the leaderboard evaluation and securing the top spot. Notably, Solar's size is less than one-sixth of Alibaba's latest model, Qwen, yet it outperforms it.
To optimize the performance of the smaller Solar models, Upstage utilized its Depth Up-Scaling method, combining the advantages of larger 13B models with good performance and smaller 7B models with intellectual limitations. By maximizing the small model's performance through depth-up scaling, Upstage successfully created a 10.7B model with an optimal combination of size and performance.
Crucially, Upstage's Solar model relied on its own built data instead of the leaderboard benchmarking dataset during pre-learning and fine-tuning stages. This emphasizes Solar's versatility for various real-world tasks in business applications, unlike models that boost leaderboard scores by directly applying benchmark sets.
Solar gained global attention by surpassing Mistral AI's Mixtral 8x7B, a unicorn with a $2 billion corporate value. Despite being smaller than Mixtral, Upstage's Solar demonstrated superior modeling expertise and optimization technology, outperforming Mixtral in evaluations.
Released for commercial use, Solar not only offers a highly usable fine-tuning model but also serves as a pre-learning model capable of self-additional learning. Scoring 66.04 points on the Hugging Face leaderboard evaluation standard, Solar surpassed Alibaba's Qwen, Meta's Llama 2, and Mistral AI's Mistral pre-trained models.
This opens up opportunities for companies to deepen their learning based on their data and specific objectives using Upstage's Solar model. It enables them to develop and implement a wide range of generative AI services, capturing increased attention.
Upstage plans to enter the global generative AI market in earnest based on its top technology through cooperation with global platforms such as AWS, Poe, and Together.ai. Upstage recently attended the 'AWS ReInvent 2023' event and announced its cooperation with AWS by explaining the process and results of building and operating a self-developed LLM using AWS's cloud services and AI platform.
Furthermore, Upstage aims to update the Solar model on Poe, a global creation AI utilization platform, allowing the public to experience the highest performance Upstage LLM directly. Poe, operated by Quora, enables users to create their own chatbot by interacting with various AI models.
Before developing the Solar model, Upstage accumulated expertise in building models across various fields, including establishing Korea's first Mathematics GPT and E-commerce private LLM. Collaborating with QANDA and KT, Upstage created the world's best Mathematics GPT, expanding LLM capabilities into reasoning, an area where generative AI traditionally faces challenges. Additionally, through Connectwave, Upstage established the first private LLM in the e-commerce industry, positioning itself as a leader in the private LLM market, catering to diverse industry needs.
Sung Kim, CEO of Upstage, said, "We are thrilled to introduce a model that outperforms global AI companies, and we aspire for Upstage Solar to set a standard for all. KT's support, facilitated through strategic investment, has been immensely valuable, and we are committed to further endeavors in the future. Our focus extends to collaborative efforts in the B2B market, leveraging the exceptional performance of the Solar model to create a substantial impact and widen the gap in our pursuits."
※ Photo caption: Korea's leading AI startup Upstage (CEO Kim Seong-hoon) is participating in the global LLM competition in earnest by unveiling its self-developed pre-learning LLM (Large Language Model) 'Solar'. The photo shows Upstage's model taking first place in the Hugging Face Open LLM leaderboard rankings.
-
Geun-Kyo Kim | Brand Communication General Director | keunkyo@upstage.ai
Seongbeom Bae | Brand Communication Manager | sungbae@upstage.aiDownload press release
-
Upstage is a leading domestic AI startup established in October 2020. Upstage stands out in the large language model (LLM) industry by taking first place on the Hugging Face leaderboard with performance exceeding ChatGPT's benchmark score for the first time in OpenLLM history. Based on these technologies, we present a reliable private LLM standard that maximizes data security and solves hallucination, helping companies conveniently use cutting-edge technology. In addition, Upstage's Chat AI 'AskUp' has over 1.4 million users, establishing itself as the largest AI service in Korea. Document AI Pack, another Upstage representative solution, utilizes AI OCR technology that has won the world's most prestigious OCR competition to automate documents by increasing efficiency and accuracy. By optimizing document processing through a pre-trained model with minimal data, cost and time are dramatically minimized compared to manual methods. Lastly, through the education program 'EduStage', we are also actively engaged in the educational content business that fosters differentiated professional talent who can be immediately put into AI business through hands-on education that incorporates AI business experience and solid AI basic education.
Upstage is comprised of members from global big tech companies such as Google, Apple, Amazon, NVIDIA, Meta, and Naver, and has participated in many world-renowned AI academic societies such as NeurlPS, ICLR, CVPR, ECCV, WWW, CHI, WSDM, and DMLR. We are solidifying our unrivaled leadership in AI technology by publishing excellent papers and becoming the only domestic company to win double-digit gold medals in the online AI competition Kaggle. While working as a professor at the Hong Kong University of Science and Technology, Upstage CEO Kim Seong-hoon won the ACM Sigsoft Distinguished Paper Award, the best paper award, four times for his research on bug prediction and automatic source code generation that combined software engineering and machine learning, and won 10 awards at the International Conference on Software Maintenance. He is considered a world-class AI guru who received the most influential paper award in 2018, and is also widely known as an instructor of 'Deep Learning for Everyone' with a total of more than 7 million views. Additionally, Upstage's co-founders include CTO Lee Tal-seok, who led Naver Visual AI/OCR and achieved world-class results, and CSO Park Eun-jung, who led the model team of Papago, the world's best translator.
