2023/02/07 | 3mins

Recently , Upstage signed the first contract to supply 'Document AI', an optical character recognition (OCR) solution, to Hanwha Life Insurance, marking the first successful launch of the 'No-code, Low-code' solution! Document AI is said to be characterized by being able to efficiently process the five documents required for insurance claims, such as medical receipts, using AI-based OCR technology, and facilitate model development and operation. [ See the highest performance AI OCR, Upstage Document AI → ]

There are hidden protagonists in the creation of Document AI. These are the people who worked on the development of 'Upstage Labeling Space', which creates meaningful learning data for the model by annotating raw data when input into Document AI. Meet Changhyun from the Document AI development team, Inha from the product design team, and Joohyun from the data team on Starview!

The hidden protagonist in the creation of ‘Upstage Labeling Space’

Do you know what Changhyun, Inha, and Joohyeon have in common? “These are the main players who created the upstage labeling space?!” Of course, that's correct. In addition, the three segments have the following things in common:

• He likes to move around and is sociable.

• With a bright personality, there are always many people around.

• If you have a good head and a goal, you spare no effort.

• He has a good sense of cooperation and values relationships with people.

• Continue to accumulate experience and wealth.

• Be flexible and have a flexible mindset.

• Even in the face of difficulties, they do their best.

  
  

These are some excerpts of the tendencies that can be seen in the twelve signs of the year of birth. It is said that the three people, who have all the good tendencies of the rat, formed the self-proclaimed 'mouse club' as stars born in the year of the rat, as fate would have it. Shall we listen closely to the story of the three people who are expected to have synergies like destiny?

Until ‘Upstage Labeling Space’ is completed

Nice to meet you. Please introduce each one.

Changhyeon: I am Changhyun Min , a software engineer on the Document AI team . I live in Jeju and have been working as a developer for about 13 years. Currently, the team is developing all parts of Document AI together with other team members.

Inha: I joined Upstage in November 2021 and am working with Joohee, Geonhee, Carrie, Dongsim, and Tim in the product design team . Since joining the company, we have renewed the website and are focusing on Document AI with the team while working on this annotation tool v2 project.

Joohyun: It has been more than 2 years since I came to Upstage. At the beginning of the company, there was no data team. I joined as a 'data manager' and worked in the NLP field along with data construction . A new data team was established in January of last year because there was a need for a team to integrate data that had been divided into NLP and OCR into one. In the early days of the team, leaders Jiyoon and I, and later Namhyeok, Lyle, Yujeong, and Cheolyoung joined to make a stronger team.

The development of Upstage Labeling Space (Annotation Tool v2) has been successfully completed.

Who were involved in this project?

Changhyeon: Annotation Tool v2 was a fairly large project considering the number of participants. The entire Document AI development team participated in the development, and Inha and Rina from the design team participated in the planning and design areas. Additionally, Joohyeon, Namhyuk, and Jiyoon from the data team actively participated in planning and functional verification from the perspective of actual users and operators of the annotation tool.

What role did each of the three play?

Changhyeon, Inha, Juhyeon: The data team identified various needs such as the use of annotation tools, necessary elements, and composition, and the planning and verification teams were concerned, and the software engineer team and Inha were in charge of production-oriented development/design. And Changhyun took the lead of the annotation tool as a PM.

Changhyun: Yes, I was in charge of backend and frontend development. On behalf of the development team, I communicated with other teams, set the specifications of the tool, and managed the overall development of the tool.

Inha: Each company/team has different rules for the role of design. I designed the foundational features, such as being offered as features and prompting users to take action. And we had discussions with the development team on whether it would be possible to implement it technically and how to implement it. Through this project, I was able to experience the overall design from planning, UI, and UX.

Hyun Joo: I participated as a guide for functional planning and design of the tool from the perspective of a user/operator. We went through the process of verifying and conveying opinions about what errors are most common to actual users, what is the most difficult for them, and what functions are needed. Thanks to you, it was an amazing and enjoyable experience to make together from start to finish!

What is an “annotation tool”?

Juhyeon: To put it simply, the annotation tool is a tool that creates fuel for the model . If you put a business registration certificate in an empty can, you don't know what it is, so if you put a little fuel for learning 'what is this' as a preliminary work, you will continue to evolve based on that. Then, even if you insert a new business registration certificate that you have never seen before, you will know 'what value this is'. So it can be said that it is a tool for making and processing fuel.

I would like to tell you about the importance of annotations. Our Document AI can encompass both general and specialized models. The general model is a model that basically reads the text inside any image. A specialized model is a model learned based on specialized, rather than general-purpose, data such as a business registration certificate. So, our OCR is basic and can handle unique cases as well.

However, when creating a specialized model, you need high-quality data so that the data can train the model. If it is a business registration certificate specialized model, a lot of business registration data must be included. Upstage's concept of Pack starts from here. Basically, the concept of Pack is that we provide what the model can do, and when the user adds training data for unique cases, it learns from it. That's why annotations become important when adding data. Usually, companies have raw data, not in the form of learning data. Annotating with the tool we created creates meaningful learning data for the model! (Read the data correctly in the beginning = Upstage's annotation tool = Upstage labeling space)

Annotation tool from v1 to v2
I'm curious about the background of the update.

Changhyun: The annotation tool is actually software that has been developed and operated internally since June 2021 to improve the performance of the Document AI model. At the time, among the development team members, Daehyun and Junyeop were developing the first version using their experience from their previous company, and when I joined the company in August 2021, Jiho joined and we all completed v1 together. After using the tool with workers in the data team, we received a lot of feedback and thought it was necessary to update the product to be included in actual Document AI. So the annotation tool v2 project started in January last year.

Joo-hyeon: (Chang-hyeon explained it well.) An annotation tool is needed to learn AI models or data, and v1 was implemented with a focus on basic functions for Document AI PoC (Proof of concept). There was no problem at all for internal use, but in order to commercialize it, one level of improvement was needed.

Inha: Yes! So, from January to April of last year, several teams joined forces to create v2.0, not v1.2.

What are the differences between v2 and v1?
What is the most focused point?

Changhyeon: There were some challenging tasks during v2 development, but raising the UI to a product level that can be sold, developing key features such as drawing tools with our own technology, and preventing mistakes while actual workers work quickly and conveniently. Finally, it was to produce all the data necessary for model learning . In fact, it can be said that I focused on all parts because it is a part that cannot be overlooked.

As a result, the OCR annotation tool became the first software created by collaboration between the development and design teams at Upstage. Compared to v1, the UI and UX were significantly improved, and by developing core technologies in-house, we were able to create a code base that could quickly develop for new requirements. Also, since the people who participated in the project actively gave feedback on the efficient and effective annotation method, detailed functions for workers have been improved a lot.

Through this, we ultimately wanted to create an easy and powerful tool so that customers could produce their own learning data using annotation tools on Document AI. While v2 focused on work convenience and meeting the requirements of the company's internal team, in the future, we will focus on seamless integration with Document AI and gradually evolve it into a good annotation tool by reflecting customer feedback when the pack is delivered to actual customers. I'm planning to go out.

What part did Juhyeon and Inha do in v2?
Did you put in the most effort?

Hyun Joo: I paid the most attention to improving usability . When it comes to actual annotations, there are a lot of parts that have to be repetitive. For example, when the information has the same value, it is convenient to put several things in a box at once. So instead of selecting one by one, we increased convenience by enabling multi-selection. There are also parts designed to reduce errors . In the tool, I set the number of dots to be taken as an even number as the annotation result. By systematically preventing odd numbers of points, the probability of incorrect data is reduced. That's how we created a defense system to improve usability and reduce data errors.

Inha: When designing, establish principles. In the case of the annotation tool, the design was to induce users to take correct actions . Annotations are easy to make errors depending on the user's learning curve, even if workers are trained. Therefore, we aimed to ensure that the tool guides the user's behavior well so that the user's small mistake is not reflected in the model and acts as a fatal error or error, and so that this does not happen easily. We tried to achieve two principles by easily and intuitively indicating what actions to take next, and preventing actions that should not be done to induce correct use.

Collaboration, how everyone goes to one place

Collaboration seems to be very important.
How was Changhyun-nim playing the role of PM?

Changhyeon: Although the development team, design team, and data team are divided on the organizational chart , I think they moved like a single project team during the tool development process. So, rather than collaboration between teams, I think we were able to proceed in the form of doing each other's work in one team. In particular, Inha participated in all scrum ceremonies of the development team (daily scrum, planning and backlog grooming, demo and retrospective, etc.).

In order for the project to flow in the right direction, quick feedback from customers who actually use it is very important, and the data team played a big role in this. When Inha and I decided on the specifications of the tool and the discussion didn't go well, we asked the data team for help without hesitation, and they always gave us good feedback. Workspace tools played a big role.

During the tool development process, we often had time to discuss together for about 2-3 hours a day, and it was often the case that the decisions made were changed two or three times within a week. This could have been a big stress, but there was a consensus that it was a process to make the best product, and there was trust in each other during the collaboration process, so I am satisfied that the project was able to proceed without major problems.

On the other hand, the reason why decisions were often reversed was because there was not enough information about the requirements or there were many stakeholders, but they all had different ideas. Especially in the latter half of tool development, there were a few cases like this as difficult functions were crowded, but the process from identifying stakeholders to collecting opinions took a long time. I'm still not sure how I can do better, but in the end, I think I need a mindset so that I can refine the process so that the iterative process of making and fixing can go a little easier, and my mentality won't be shaken even with frequent changes. And the belief that the product gets better seems to be the most important thing!

Inha and Joohyun collaborated
Did you have any difficulties or concerns?

Inha: When creating the tool, the most difficult part was not being able to satisfy all stakeholders. There are users who actually work on annotations, there is a data team that is an administrative admin, and there is a model team that models the final convened data. Each of you may have different expectations for the final output you want to create. Nevertheless, it is necessary to coordinate well with various stakeholders within a limited time, and to make decisions by making choices.. I think that was difficult.

Joohyun: Last year, when I was working on the project, one of my OKRs was 'Let's prepare a v2 channel and create a quick application process'. In v1, it took a lot of work because there was no channel that could collect the data-related voices of the annotation tool well. So, this time, actual workers wrote errors in the bug report from time to time, and the data team and development team checked the file together. This process reduces the resources of the decision-making process. In the future, I want to set up a 'QA bulletin board'.

I'm also curious about what was memorable during the collaboration process.

Inha: Every two weeks, we had a demo session (time to share the overall development progress and each person's situation) . It was great to have a separate retrospective session with the development team after the demo session. At first, I wrote a retrospective with lines like 'thank you', and then Jaeho made various activities. For example, 'Draw my biorhythm for a week' is memorable. What happened at this time, what was wrong and what was regrettable... It was a place to draw these things in a biorhythm graph and reflect on each other. In fact, I was very tired at the time. However, when I had honest retrospectives with good colleagues, I naturally felt refreshed.

really! There's a story behind the story. Right before the homepage release, there was a period of overlap between the annotation tool and the schedule. There was an 8-hour meeting every day for almost a month, and since the annotation tool is actually functionally complex, at some point I thought, 'Is the process so hard because I can't do it?' Since it's what I've been doing, it's time to work with my mind at ease, thinking that I should finish it well, and at that time, Changhyun-nim said, "Isn't it easy, Inha?" “The annotation tool is not as easy as I thought, but it seems to be difficult. When I think about it when I was a freshman in society, I think my mentality always came first. But you have to rest well, it will work out.” At that time, those words were so reassuring and encouraging!

Attitude to Practice the Upstage Way

On the usual upstage way
Are there any items you consider most important?

Changhyun: What I think is important while working is the 'One team' spirit for smooth collaboration. So I think it's important to create an atmosphere where you can work happily based on mutual trust. To make it a natural atmosphere for anyone to ask questions and express their opinions freely!

Joohyun: For me, 'one team' is something I naturally practice and always consider important. I think the 'one team' is important not only to the team you belong to, but also to multiple teams. In my work, there are many decisions that need to be made, and this affects each other's work, such as the development team and the model team. However, making decisions within limited resources makes it difficult to reflect all needs. That's why it's important to work as a team, and I think there must be honest communication for this. In the development of the annotation tool, I thought that the data design would be much easier if the data team knew exactly how to use the tool and clearly conveyed the desired requirements. I think I gave my opinion without hesitation before considering conditions such as development time and UI implementation. Nonetheless, Inha and Changhyun listened attentively with the attitude of 'Let's talk about it first', and as honest opinions were exchanged over and over again, I think I was able to bond and participate in the project more fun!

If any of the items of the Upstage Way are difficult to practice,
Are there any parts you are worried about?

Changhyeon: In a book I read recently, I saw a sentence that said, 'The purpose of a team is not to achieve goals, but to match goals', and I think it is very sympathetic and important. So I want to share enough information so that I can reach the final decision through intense discussion, and I want to do well in the part where the decision is well recorded and recognized by people. I want the entire product-making team to have a good understanding of the project's current situation and to have a consensus on what needs to be done to move forward.

In this project, I spent a lot of time discussing with the design team and data team, and looking back to see if all the time was efficient, I am thinking about how I could have done better. As an attempt, I wrote a weekly document for the Annotation Tool Project for about a month in March, and it was very helpful in organizing and sharing the issues discussed and decided during the week and the current status of the tool project. We plan to make this kind of attempt in earnest in the development of OCR Pack.

Inha: For me, it's 'Sharing'. Every time we share information through scrum, etc., but it seems difficult. I was worried about the part of 'How far should I go to share'. When there are matters from 1 to 10 in the overall area of design and development, it was about how much to share from 1 to how much sharing would be sufficient. As the development speed started to pick up, naturally sharing with the development team was active, but I think our team had difficulties in aligning the detailed history. I want to find a good direction for effective and efficient sharing in the future and keep trying.

Future plans and gratitude

Do you have any future plans or personal aspirations?

Changhyun: We plan to focus 100% on developing Document AI . I think we have gone through enough preparations to make the product a reality. Now, we have to work hard during the remaining period to make it and put it on the market as the company's flagship product. Personally, I want to expand my capabilities into the areas of product management and people management. Many engineers are afraid or reluctant to step into the management field, but personally, I think management is really important for an organization to grow in the right direction, and I want to study more in this area and continue to gain experience.

Inha: I’m really excited to see what Upstage’s first product, Document AI, will look like. I also plan to focus on Document AI design and design along with Changhyun. In particular, I will be in charge of the data management department when it comes to pack design. And with the development of annotation tools, we plan to continue HCI (Human Computer Interaction) research ! People have a huge influence in the data collection process. How we can handle this in terms of data quality and efficiency is important, but interest from industry and academia is not that high yet. Since I have been conducting research in the HCI field, I often look at the interactions and processes themselves when computers and people interact. We want to permanently solve this problem we actually face, and to this end, we are regularly holding research meetings with members of the data team. Cheolyoung is also someone who used to do HCI, so me and Cheolyoung meet once a week, and once every two weeks, the four of us, Joohyun and Namhyuk, conduct a study for overall development. I hope to achieve good results within the year through continued research and study!

Hyun Joo: I am working with a focus on building/designing data. While working with Inha, I came to think a lot about HCI, which views the process of creating this data as a meta. From that point of view, when data is created, I have to reconsider not only the technical level of problems, but also various other intertwined factors. Thanks to Inha, I think my horizons have broadened. And if I get a chance this year, I want to work in a good place like Bangkok. Opportunities to go abroad to work are not easy, but I want to try Upstage because it is possible to work anywhere.

If you have anything you want to say to the stars of Upstage, please ask.

Joohyun: I couldn't tell Changhyun directly, but thanks to him, I was able to work on the project while expressing my opinions in a really fun way. In the future, I think the data team will say something like 'please add this function'. I would like to say thank you in advance and wish you well.

Inha: I also posted it on the #all-thanks channel on Slack to thank the stars, but there are so many people involved in this project. Directly, they are the development team and the data team, but there is also a model team, and Rina from the BX team is also nearby. It was great to be able to work with so many people. The fact that the development team took care of it was a lot of comfort to me. The heart itself that always recognized me first was very warm and I was grateful. The annotation tool is also a good output, but I think the experience I gained from working with many people was very good. And I think my love for Upstage grew even more while working with these people!

Changhyun: I also want to thank everyone! Like the ending credits, I want to express my gratitude. To upstage stars~! And Special Thanks to. Lee Jun-yeop, Nam Dae-hyun, Kwon Eui-hyeon, Lee Jae-ho, Lee Ji-ho, Moon Soo-young, Baek Soo-young, Park Mi-seong, Cha In-ha, Oh Joo-hyun, Kim Nam-hyuk, Han Ji-yoon

(This interview was conducted last year, but will be released in 2023 in line with the launch of Document AI. ‘Upstage Labeling Space’ is said to be working on renewal in the first half of this year, so we are looking forward to the performance of Document AI, which will be applied to practice in various industries in the future. please!)

[ See the highest performance AI OCR, Upstage Document AI → ]

Event

Create new value through capitalizing data

Feel free to test the Document AI API in the Upstage console and create the service you want!

←Back to Blog