Image source: Generated by Unbounded AI
Source: infoq
Author: Li Dongmei
Combined with the underlying technical logic of ChatGPT, there are roughly four directions for the industrialization of ChatGPT in the short to medium term in the future: intelligent customer service, AIGC applications in text mode, code development related work, and image generation. Among them, the most suitable project for direct implementation is the work of intelligent customer service.
The intelligent customer service based on the large model technology is fundamentally changing the traditional human-computer interaction process. The large model automatically generates the dialogue process to make the operation of the intelligent customer service more efficient, which can improve the complex problem solving rate, human-computer interaction perception, and intention The efficiency of operational content such as understanding, process construction, and knowledge generation.
From the perspective of product penetration rate alone, smart customer service has been gradually popularized in e-commerce, finance and other fields as early as the past seven or eight years. The two core changes brought about by the large model, one is that the cost of developing intelligent customer service products has been greatly reduced, and the other is the improvement of user experience.
So, if you want to combine the LLM large language model with intelligent customer service products, or put the former in the field of ToB SaaS application software, how should you start building a technology stack? How will large-scale model products empower intelligent customer service products? In this issue of “A Date with Geeks”, we specially invited Xu Wenhao, the founder of bothub, the founder and CTO of Buqituo Network Technology, as the moderator, and Jia Haowen, the director of computing technology of Huayuan and the co-head of the digital human business department, and Zhongguancun Kejin Wang Suwen, director of intelligent interaction research and development, head of Zhongguancun Kejin intelligent customer service technical team, and Wang Chao, head of JD Yunyanxi KA product, discussed the implementation and future development trend of AIGC in intelligent customer service products.
The following is the transcript of the interview.
***Xu Wenhao: Audience friends, hello everyone! Welcome to InfoQ “A Date with Geeks”. Today’s topic is the special planning of “AIGC Publication of Tiangong”. We hope that through this plan, everyone can fully understand all aspects of AIGC in the field of intelligent customer service, and deeply perceive this change. ***
***In this program, we have invited three guests to discuss the application of AIGC in the field of intelligent customer service. Overall, we will divide the discussion into three parts. The first is the application of AIGC large model in intelligent customer service products; the second is the deployment of AIGC architecture and the design and selection of tool applications in intelligent customer service; the last is the method of building a high-quality dialogue system. ***
***The three guests today are all experienced experts in the field of intelligent customer service and intelligent interactive products. The first guest is Wang Chao, the person in charge of JD Yunyanxi KA product. The second guest is Mr. Wang Suwen, director of Zhongguancun Kejin Intelligent Interactive Research and Development. The third guest is Mr. Jia Haowen, Director of Computing Technology of Huayuan Academy and Co-Head of Digital Human Division. ***
***Let’s start with the first question, I am very curious: What changes has the emergence of AIGC brought to intelligent customer service? I would like to invite Mr. Wang from JD Cloud to share your views. In your observation, what innovations has the emergence of AIGC brought to intelligent customer service? ***
Wang Chao: The emergence of AIGC has aroused widespread concern in the entire field of intelligent customer service, and prompted relevant peers to conduct a lot of exploration. Regarding the cognition and future changes of intelligent customer service, these cognitive changes are changing with each passing day.
The JD Cloud Yanxi team has been paying close attention to the progress of intelligent customer service applications at home and abroad. In addition, the Yanxi large-scale model we are developing will be released in July, and we are also continuing to conduct various large-scale model experiments in the customer service business. In recent months, my personal understanding of AIGC has been completely different from that of 3 months ago, so the views I want to share today represent more personal opinions and current views.
The impact of AIGC on intelligent customer service can be viewed from two levels. First, from the perspective of common management problems and technical difficulties in our industry, AIGC has the potential to solve them. We all know that large models will greatly improve the response level, anthropomorphism and service experience of intelligent customer service, and can greatly reduce operating costs.
In terms of robotics, I don’t think it needs to be discussed too much, because we have already focused a lot of attention on robots. What I want to say is that the large model is used in a wide range of intelligent customer service fields, especially the verification information in the direction of intelligent customer service management. For example, in our practice of intelligent assistance, some previous technologies have recommended some key points and extracted key information in conversations. Although there are solutions, it is usually difficult to achieve a good balance between cost and effect. However, through the verification of the large model, we found that it has good solving ability and potential in dealing with these problems. In addition, for quality inspection work, methods such as keyword regularization or intelligent quality inspection methods commonly used in the industry have certain effects, but the accuracy rate is often low, the workload is also heavy, and the period for improving the accuracy rate is long. .
However, by practicing with large models, we found that it works well for understanding abstract QC criteria and performing QA work. In addition, in terms of employee training, we have seen some examples of leading banks introducing dialogue robots for training. In addition to the above, we also conduct business analysis in the customer service center, and need to summarize information on customer needs, customer portraits, and risks in customer consultation. In the experiment, the effect of the large model is also very good. I think that in the field of pan-smart customer service, the application potential of large models is huge. We believe that in about half a year, mainstream products on the market will usher in an important upgrade.
Based on what is currently understood about the industry and domain, we can explore whether the broader domain is about to change. For example, what changes can it bring to customer service and customer service? I think this question could be more challenging, and at this stage, no one can pinpoint specific changes. But we believe it will make a big difference, at least in some directions, such as proactive services. For example, e-commerce companies often carry out various activities, and the work of promoting and explaining the activities usually does not fall to the customer service center. Because it is very difficult to carry out such a wide range of activities by manual or traditional machine methods. But we can imagine a future where our customer service can explain many activities well through the introduction of a large model and the input of basic information, which represents the possible potential of future customer service in terms of active transformation and upgrading.
***Xu Wenhao: Let me summarize, in fact, the emergence of AIGC has expanded the scope of applicable intelligence in the field of intelligent customer service. Not only in the traditional intelligent field of answering after-sales questions, but in all aspects of the entire customer service link, the large model can be applied, and it can even be extended to the marketing field. Teacher Wang Suwen, what kind of phenomenon do you see from your perspective? ***
Wang Suwen: As mentioned just now, the scope of intelligent customer service is very wide. Technologies such as intelligent customer service and robots can actually change the traditional human-computer interaction process. Through the use of large models, especially the use of automatic generation of dialogue processes, the operation of traditional intelligent customer service can be made more efficient. In the traditional way, we usually need to configure the knowledge base manually, but the effect is not obvious. However, the ability to solve complex problems and the direct answer rate of questions can be directly improved through the automatic generation of dialogue processes through large models, which is a significant disruptive impact.
The second point is that large models can also achieve the goals of reducing costs and improving efficiency. From the perspective of intelligence, labor costs have always been relatively high, because it requires human-assisted machines. Through the application of large models, it can assist in improving the efficiency of knowledge base construction and operation, thereby achieving a significant effect of cost reduction and efficiency increase.
The third point is about the anthropomorphism and user experience of the robot. Traditional robots often fall short in these ways. However, the emergence of large models makes the dialogue smoother, more anthropomorphic, and more like human-to-human communication. This is a very powerful disruptive effect.
Of course, for general customer service, including quality inspection, assistants and sparring, the large model will also have a corresponding subversive impact, and different products will have different impacts.
***Xu Wenhao: I noticed that Mr. Jia is the co-head of the Digital Human Division. From your point of view, if the combination of intelligent customer service and digital human will bring some new innovations? ***
**Jia Haowen:**Before answering this question, I would like to go back to the field of intelligent customer service. The development of intelligent customer service can be traced back to the history of artificial intelligence, which is based on the accumulation of language. I remember about ten years ago, there were several floors filled with customer service people who spent most of their working hours sorting out user issues. At that time, even the tools such as the knowledge base mentioned by the two teachers were not very complete.
Looking back now, Alipay and other customer service products have gradually expanded from the initial knowledge base to later senior assistants, and derived tools such as the Rasa framework for simulating multiple rounds of dialogue scenarios. However, as the two teachers said just now, all these dialogues are still lacking in personification. All previous customer service tools essentially failed computer industry standard tests. But when big models come along, especially things like ChatGPT or Vicuna, they have a level of personality, albeit perhaps at the level of a 10- or 11-year-old kid. In a way, they are able to pass the Turing test.
Now going back to your previous question, the combination of intelligent customer service and digital human will bring some new innovations. In the initial stage of the digital human field, anthropomorphic Q&A with humans based on knowledge bases and large models cannot be completed in real time. However, with the expansion of technology, we have been able to achieve anthropomorphic figures, combined with large models to simulate real scenes. When we apply it to scenarios such as knowledge bases, customer service, and digital employees, the field of digital humans may face two challenges. The first is the completeness of the data. Although ChatGPT and the like look cool, they are essentially generated based on past historical data, similar to the generation of cloze. While this approach increases efficiency and reduces scripting costs, the quality of the generated content is virtually uncontrollable. In the output content of comprehensive fields such as digital human and intelligent customer service, we cannot be completely sure of the generated content. Therefore, we may need to introduce different verifications and safeguards to ensure that what a digital human or ChatGPT etc. say is more like a human speaking than gibberish.
To sum up, although the large model may be relatively popular now, and the future prospects are also bright, it is still in a state from 0 to 1 at the current stage. In the future, we may expect it to grow into a towering tree, but now, whether it is for the entire industry or everyone’s expectations for it, it is still necessary to let it grow gradually in a better soil, and truly be able to provide intelligent customer service. , Bring efficiency improvements to fields such as digital humans.
***Xu Wenhao: I want to explore this issue in depth. I would like to ask Mr. Jia, according to your point of view, if we introduce a large-scale model in the field of intelligent customer service, will it bring benefits? I mean in terms of the current situation, regardless of the development in two or three years. Just looking at it now, is there a benefit from the perspective of your customers or your internal product from the point of view of bringing in the big model? ***
Jia Haowen: Whether to introduce a large model depends on specific business scenarios and user needs. For large companies such as Ant Financial or JD.com with large and mature systems, blindly introducing a large model may increase additional costs. We need to take some compatibility measures to avoid the uncertainty caused by AI-generated content. For smaller companies, using large models as a supplement to their knowledge base, with some level of human review, can be a good way to increase efficiency. It can help intelligent customer service to better understand the user’s semantics, improve the quality of the knowledge base, and even improve the user experience. In addition, for higher-level applications such as cognitive intelligence, the introduction of large models may help to better understand and recognize users. In the Internet industry, there is an important concept called “thousands of people, thousands of faces”, which means that each customer service can provide personalized services. Therefore, we need to look at whether to introduce a large model from a dialectical point of view, and make decisions based on specific circumstances.
***Xu Wenhao: Mr. Jia’s point of view is that large companies need to be cautious in introducing large models. The introduction of large models may increase costs and increase various risks, and they need to rely on the original methods or models as a back-up measure. Then I would like to ask Teacher Wang Chao, has JD Cloud customer service introduced AIGC? What will be the benefits after the introduction? ***
**Wang Chao: **For large companies to cautiously apply large models on C-end products is out of consideration of ensuring customer experience and service security. In a consumer-facing business, caution is critical. In this regard, both validation and experimentation are proceeding with great caution.
In the field of business-oriented or operation-oriented, you are more “brave” in experimentation and verification. We introduce large-scale models to help operations build copywriting and scripts, which provides good support for operations to solve creativity and efficiency issues. In addition, it is also very active in the direction of employee management such as quality inspection, assistance and training.
Is the large model “just needed” or “follow the trend”?
***Xu Wenhao: How much can this increase the efficiency of ordinary employees? Have you measured it, or have some specific data? ***
Wang Chao: We have not yet made specific calculations on this problem, because the application of large models behaves differently in different work projects. For example, it could be several times faster at generating scripts and copywriting. However, in terms of daily analysis and quality inspection construction, the effect may vary from case to case. Therefore, at the current stage, it is difficult for us to quantify and draw an accurate conclusion, but we are sure that the application of large models has indeed improved efficiency.
***Xu Wenhao: Mr. Wang Suwen, have you introduced AIGC to your customers or products? Can you see specific benefits? ***
Wang Suwen: In our business field, we mainly target the ToB market, and we serve customers in various fields, such as the financial and insurance industries. These customers have several needs in terms of innovation, including incentives to reduce costs and improve efficiency. We have carried out some co-creation cooperation with customers and verified them. For example, we created a marketing assistant for a client that automatically generates some copy. Traditionally, the level of each employee is uneven, including the lack of standardization and unity in terms of marketing skills. Through our marketing assistant, we can first help them generate a unified copy and copy it based on the excellent experience in history. The second is to reduce costs and improve efficiency. They no longer need to spend too much time on training, learning, and memory. Our large models are widely used in industry.
We have also developed telemarketing bots that can answer some questions directly. Of course, we have to take compliance issues into account, including complying with relevant laws and protecting data security. We are working with Xinchuang Institute to work on solving these standardization issues.
***Xu Wenhao: Everyone is really trying to use the ability of the big model. However, there are still many specific problems to be solved, especially the challenges of controllability and safety. From the perspective of smart customer service, the e-commerce industry was the first industry to introduce smart customer service, because there are large-scale promotional activities such as “618” and “Double Eleven”, and the traffic peaks during these activities are very huge. ***
***In the past few years, domestic service providers have also developed various smart customer service products for various financial institutions such as banks, insurance and wealth management institutions. In this case, it appears that financial institutions will be left behind if they do not use robo-advisors. However, institutions such as bank customer service do not have peak periods like “618” and “Double Eleven”. Is the introduction of smart customer service by these institutions out of necessity, or out of “following the trend” or a sense of crisis? If it is a rigid demand, where did this demand originally come from? ***
Wang Chao: Driven by the wave of digitalization, it is easier for financial institutions to realize the implementation of intelligent customer service. In the financial field, intelligent customer service has been widely used and has become one of the more common products of financial institutions. For financial institutions, the core needs of intelligent customer service are mainly to reduce costs and improve efficiency, which can liberate human resources from tedious work. Especially in business scenarios such as automatic question and answer, notification and return visits for some high-frequency questions, intelligent customer service can save human resources and enable them to focus on more professional and creative work instead of repetitive, frequent and low-value work. Let them have resources to invest in developing and maintaining high-net-worth clients, and focus more on performing more valuable tasks. In addition, with a new generation of basic breakthroughs, such as the emergence of large models, intelligent customer service can also obtain powerful natural language generation capabilities, making it more intelligent and efficient.
***Xu Wenhao: So it is a rigid need, because there is a lot of repetitive work to reduce costs and increase efficiency. Mr. Jia, are your customers in the financial field or other fields paying attention to smart customer service? ***
**Jia Haowen: **In addition to the financial field, there is also a demand for large models in vertical fields such as insurance, law, and health. Especially from a corporate perspective, the need to generate and extract knowledge is becoming more and more apparent. When we talk to banks or insurance companies or discuss issues with legal practitioners, the information they provide is factual in nature. Through the large model, we can better identify the intention of the user’s question and perform clustering or splitting.
As Mr. Wang Su said, industries such as finance, insurance, and law have great potential for improving efficiency. This is a process of reducing costs and improving efficiency. From my personal point of view, this is a rigid need, especially in terms of efficiency. However, these industries may not take an overly aggressive approach, as businesses related to finance, insurance and law are highly sensitive and require a strong degree of specialization. In this regard, another issue arises, which is the maturity assessment of the results generated by large models. Although large models like GPT-4 perform well in some professional exams in the United States, it is still a problem whether it can pass the corresponding exams in China, especially in a large and complex semantic context like Chinese, which may require further research. Research. In this regard, our company plans to cooperate with Zhejiang University in July to release a large model of the legal vertical field to provide better solutions for this specific field.
***Xu Wenhao: Large models in the legal field are similar to those in the financial field, and the requirements for the quality of generation are very strict. In the legal field, a small mistake may have a negative impact on the consumer experience, and even cause losses to the merchant (B-side), which requires higher accuracy and reliability of the generated results. On this question, what does Teacher Wang Chao think? ***
Wang Chao: I think whether it is e-commerce or the financial industry, the requirements for intelligent customer service are very strict. In addition, for e-commerce smart customer service, the big promotion period and daily consultation are very impressive. In the self-operated business of our team in JD.com, the daily consultation volume accounts for 70% and 90% during the peak period. This 70% of the consultation volume is already of great value. I believe this logic also applies to various institutions such as banks. We have also noticed that the intelligent customer service capabilities of some banks are still insufficient. Therefore, we need to think about how to improve the maturity of intelligent customer service, and it needs to adapt to the development stage of the enterprise or bank. At present, many banks are in the initial stage of intelligent customer service, mainly focusing on FAQ and simple multi-round dialogue construction, while service capabilities and user experience may not yet reach a satisfactory level.
Based on our experience in the field of e-commerce, we believe that the bank’s intelligent customer service needs to be further developed, especially in the operating system, which needs to be greatly upgraded. For example, we require grass-roots customer service to be close to artificial levels in terms of service standards and skills, which has a major impact on the reform of our operating system. However, in the process of cooperating with banks, we found that many banks face challenges in the construction of intelligent customer service. The cooperation between technology suppliers and banks is more similar to the relationship between Party B and Party A, which is different from our cooperation model. There is still a gap between banks and our requirements in terms of organizational structure and personnel training. We spoke to many of our banking partners and found it common that they agreed with our development methodology but also found it difficult. Pushing forward internal reforms is complex for banks and requires more effort. I believe that the large model is an opportunity, because it can reduce the operational difficulty of intelligent customer service and simplify the organizational structure. I am looking forward to it.
***Xu Wenhao: I believe that the big model is a huge opportunity for everyone here and most of the people in the field of intelligent customer service and natural language processing. We share a similar view that large models can help us achieve more anthropomorphic and personalized conversational experiences. However, when it comes to providing services to C-end users, whether it is e-commerce or banks, everyone will be more cautious. We all want to ensure that the final output is controllable, whether it involves a 10,000 yuan certificate of deposit or a 1,000 yuan order, this is something that needs to be taken seriously. ***
*** Just now we also talked about another important topic, user experience. We can observe that the previous generation of intelligent customer service and dialogue robots, their answers are pre-written templates, for example, the answer about the delivery address is usually fixed, with only minor changes. There is a certain gap between the answer of this fixed template and the real human customer service. Now everyone is working hard to improve the anthropomorphic level of dialogue robots, so that the intelligent customer service behind them can be more emotional, better able to understand user emotions, and perform multimodal calculations. In this regard, have you invested in research and development? ***
Wang Suwen: I think it can be discussed from three aspects: personification, humanization and individualization.
The first is anthropomorphism. In the field of intelligent customer service, anthropomorphism has always been a pain point, because the performance of traditional robots in this area needs to be improved. We have been exploring how to build anthropomorphic dialogue interactions. This involves how to design situational dialogues, how to disassemble problems, how to inherit context and how to understand multiple rounds of dialogue. Overall, we hope that bots can provide more relevant and natural modes of conversation and interaction.
Followed by human services. On the basis of accurately identifying the scene or intention, we also need to further improve the service on the basis of anthropomorphism. I think multimodal affective computing is an effective way to achieve this goal. For example, our company has developed a virtual digital human customer service, which can conduct human-computer interactive dialogues, combined with emotional computing, and recognize user emotional expressions through multimedia methods such as video, voice, and text. In this way, intelligent customer service can give corresponding emotional feedback to users, create human-computer interaction with emotional understanding and warmth, and realize more humanized services. In terms of affective computing, traditional methods have two modes: rules and machine learning. Through machine learning, we can train models to automatically learn emotional states and achieve classification standards, so as to better adapt to different fields and contexts. In this way, better emotional expression effects can be obtained and more humanized services can be provided.
Finally, there is personalized service. We need to implement personalized services based on the user’s portrait to achieve the effect of “thousands of people with thousands of faces”. For example, we have developed a user insight platform, which can mark users and accumulate portraits through basic user portrait information and insight analysis during historical conversations. In the follow-up dialogue, we can provide different dialogue processes, reply methods and recommendations according to the user’s portrait, so as to achieve more accurate personalized services and improve the satisfaction of users and enterprises.
***Xu Wenhao: Teacher Wang Suwen provided a lot of valuable experience sharing, which can be used for reference and imitation by those engaged in intelligent customer service work. JD.com is a platform with a large number of users and products. Everyone hopes that when using smart customer service, there will be a feeling that there is a real person behind it. I would like to ask Mr. Wang Chao, what work has JD.com done in R&D and products to realize the ability of “thousands of people, thousands of faces”? Is there any experience to share with you? ***
Wang Chao: We have made a lot of investment in improving experience and personalized service. The emotional intelligent customer service developed by the Yanxi team is the industry’s first large-scale commercial emotional intelligent customer service. Since 2018, we have introduced the ability of emotion recognition and response in the robot response ability. This technology is not only used in the field of customer service, but also in quality inspection and personnel management services.
Answering this question involves two aspects. The first direction is technology, such as cutting-edge technologies such as multi-round dialogue, which are the main development directions of current intelligent customer service technology. Another important direction is operation, that is, how to conduct detailed scene dismantling, analyze human-machine differences, and realize automated problem discovery and comparison of human-machine service differences through monitoring and tool systems through benchmarking with human customer service. In this way, we gradually realize refined crowd service experience analysis from the overall difference analysis of the overall experience and crowd service experience. With such a system, we can continue to optimize the overall service experience, and finally realize the machine service coverage of JD.com’s current daily machine reception volume of 70% and 90% during the promotion period.
**Jia Haowen:**The theme of tonight’s live broadcast is about the understanding of digital human customer service and large models. Our understanding of traditional customer service, whether it is manual customer service or FAQ, can be regarded as low-end intelligent customer service. As users, we expect customer service to be competent and able to provide assistance in solving problems. At the same time, if the customer service can show anthropomorphic characteristics and provide personalized services, the user experience will be better.
In the field of large models, anthropomorphism and personification are very important. Our company is more inclined towards the research direction of cognitive intelligence and psychology. We focus on how to quickly obtain users’ psychological tags, and judge users from a psychological perspective, so as to provide better services. We can imagine a scene, such as Jarvis in the Marvel movie “Iron Man”. It would be an ideal state if the big model in the future can be like an assistant that can give the most appropriate answers and recommendations according to our mood and preferences that day.
At present, we are still in the preliminary stage in the research on the combination of large models and digital humans. However, with the emergence of academic research results in the field of multimodality, some better and lower-cost experience products may emerge.
***Xu Wenhao: Everyone mentioned the pursuit of emotion, and it seems that everyone is moving in a direction closer to real people. Most of the audience in the live broadcast room are students engaged in technical work. We hope to share some experience on how to build a high-quality dialogue system, whether it is intelligent customer service, after-sales service or pre-sales shopping guide. Where should I start when entering this field? ***
Jia Haowen: If we want to build a high-quality dialogue system, it actually involves the issue of traditional Internet project approval. In this process, we need to consider the selection of business product architecture, technical architecture, and the actual landing form of the product. At the same time, we also need to take into account that many large enterprises or medium-sized enterprises already have many existing customer service products. If we want to improve the capabilities of these existing customer service products through large models, we may need to adopt a more conservative approach. For example, large models can be used as an external knowledge base tool for providing knowledge input. Going back to the architectural aspect just mentioned, both product architecture and tool selection are important considerations. For the initial entry point, tools such as keyword tagging systems and traditional regular expressions may be involved. However, there is no general standard paradigm for the specific choice of this paradigm, because it is related to the characteristics of each business.
***Xu Wenhao: From the perspective of practitioners, whether it is a smart customer service company or a smart customer service SaaS or cloud platform, they can think about how to further improve the dialogue system to provide higher quality services. Although we have discussed a lot about big models today, in fact, when it comes to traditional intelligent customer service, whether in banking or e-commerce, consumers will more or less feel that there is no real person behind them to provide services. We need to think about how to improve the user experience and invest more effort to improve the status quo. ***
Jia Haowen: If we narrow the scope of the problem and focus on providing more efficient and high-quality intelligent customer service content output, then we can regard the large model as an enhanced dialogue service. In intelligent customer service products, what users expect to encounter is an intelligent, empathetic, and communicative robot. At the same time, they hope that the content of the answers can focus on specific business areas, such as customer service marketing and other scenarios. In this case, we can combine the intent recognition, dialogue flow and multi-turn dialogue capabilities of the large model with external data sources such as traditional FAQs. This means that we need to converge the entire language processing process, such as summarizing visitor question data and comparing it with user questions, and even inputting a large amount of privatized data while users are asking questions, so as to supplement traditional intelligent robots, voice robots and content Capabilities of external knowledge bases. Through this combination, the efficiency of the user experience can be significantly improved in a short period of time. Further, we can consider the improvement of anthropomorphic capabilities, but for companies that want to upgrade their dialogue system capabilities, the challenges may be higher, so it is recommended to introduce them carefully. As an entry point, considering large models as part of external knowledge input may be a low-cost and fast entry point.
How to improve the dialogue quality of large models
***Xu Wenhao: Teacher Wang Suwen, if we want to build a high-quality dialogue system to improve the current situation, in which areas should we invest in research and development? ***
Wang Suwen: We can consider the following points to improve the quality of the dialogue system.
-
Reduction of data labeling costs: Traditional data labeling methods require a lot of manual labeling, which will bring time and resource costs. In order to reduce this dependency, we can study how to use a large amount of unlabeled data for unsupervised training, thereby reducing the need for manually labeled data. Such an approach can improve the efficiency of data acquisition and reduce costs.
-
Improvement of generalization ability: It is not enough to answer a single question. The dialogue system needs to have a certain generalization ability to adapt to different scenarios and user needs. By learning the diversity and regularity of language, we can improve the generalization ability of the model so that it can deal with more problems and situations.
-
Construction and selection of dialogue models: When choosing a dialogue model, we need to consider the applicability of the model in different scenarios. There are currently many large models to choose from, so we need to choose a suitable model according to specific needs to achieve higher accuracy and effect.
-
Continuous learning and optimization: The dialogue system needs continuous learning and optimization, because the effect of the initial online model is not perfect. The system should have the ability of self-iteration and self-optimization, and gradually improve the effect and performance through continuous use and feedback. This process of continuous learning and optimization can meet the needs of customers and make the system more and more intelligent and efficient.
***Xu Wenhao: Teacher Wang Chao, do you have any comments to add to the previous discussion? ***
**Wang Chao:**I think the key to this question is that from the perspective of customers and business parties, as well as our perspective as platform product developers and designers, they all point to the same goal. No matter from which point of view, we need to pay attention to the core needs of the business and the expectations of the intelligent customer service system in providing services. Under different service forms and models, the technical architecture of the product and the technical capabilities introduced may be different, but overall there is not much difference.
For example, for some business parties, they may hope that robots can provide basic question-and-answer and information query capabilities. At this time, we may only need to provide some FAQs and some simple dialogue tools and algorithm models to meet the needs. For some business parties, they hope that robots have the ability to handle and follow up business on their behalf, and even provide scenario-based dialogue services and full escort. In response to different levels of customer needs, we need to design products and build technical architecture accordingly, and introduce corresponding capabilities. Therefore, I think it is very important to follow up with the business side, help the business with consultation, and gain a deep understanding of their business. According to different customer needs, carry out product design and technical architecture construction, and introduce corresponding capabilities to meet their needs.
***Xu Wenhao: Which model has the better effect after you try it? What necessary tools and applications are needed, how to choose the architecture, etc.? ***
Wang Suwen: When using ChatGPT or similar large models, you can apply and deploy them according to the following steps.
-
Model training and tuning: Choose a commercially available open source large model, such as Zhipu ChatGLM, Baichuan large model, etc. Verify and test the performance of the model according to your own needs and business. Collect domain-related data, and use these data to conduct domain training on open source large models, and can also perform semi-automatic generation of instruction sets. By fine-tuning and screening multiple rounds of dialogue data, the dialogue ability of large domain models is enhanced. Make sure the model meets requirements in terms of safety, fine-tuning and post-processing according to specifications and values.
-
Model engineering and performance optimization: For generative models, consider the speed, capacity and compression of model reasoning. If the model is too large to be accommodated by a single card, parallel reasoning with multiple cards on a single machine or multiple cards on multiple machines can be considered. Perform performance optimizations on models, including depressurization, compression, and acceleration for better performance.
***Xu Wenhao: Teacher Wang Suwen, do you have any recommended Chinese basic commercial models? ***
Wang Suwen: I believe that everyone will make trade-offs according to their own needs and standards when choosing a model. Each model has its own characteristics and advantages. In our business requirements, we tested multiple models, and finally chose the two large models of Zhipu and Baichuan, because the large models of Zhipu and Baichuan have reached a certain maturity in commercialization. They recently released a new model, which also shows that they are constantly optimizing and iterating. I believe that with the continuous improvement of these models, the development of domain models based on these large models will bring better results.
***Xu Wenhao: In terms of testing large models, does Mr. Jia recommend any tool architecture, or any particularly critical applications? ***
**Jia Haowen:**For small and medium-sized companies, it may be difficult to develop from scratch or perform instruction set tuning on existing models. Models are often too large to fit even on a single graphics card or machine. In addition, collecting structured data, especially data related to specific business domains, is also critical. Because in the original training process of ChatGPT, a lot of data collection and organization were done, which requires the use of data in its own field to fine-tune the model in the process of instruction set tuning. This involves some multi-machine and multi-card parallel computing, which may require algorithm and model trainers to have a high level of knowledge, such as tensor acceleration measures and gradient accumulation measures.
In the process of deploying and operating models, it may be necessary to consider network speed for model training, hard disk selection (such as Zata or SSD), storage, and data transmission acceleration tools, which have high requirements for the operation and maintenance environment. Generally speaking, the training process of the current large model may be relatively difficult, but for the simple deployment and inference process, it is basically feasible to deploy it on the V100 based on a large model such as 6B or 13B.
If the model tuning and training is completed and deployed to the online system, we usually consider upgrading the entire architecture. At present, the popular one in the industry is the Milvus vector database, which can intermediately cache the generated results through vector retrieval, similar to the Redis cache we usually use. Because of the characteristics of the cloze generation mechanism, although the specific style of each generation may be different, the general meaning is the same. In order to reduce online costs, we can adopt such a mechanism. At the same time, a complete set of solutions is also required for the content review system, training data preparation system, and labeling system.
In general, training a large model is not necessarily terrible, but it may increase the requirements of our previous technology stack, but this improvement can also be overcome, but it may be a little difficult, but we can completely learn and practice to meet these challenges.
***Xu Wenhao: Model training itself is only a part of the whole process, and surrounding supporting measures are also crucial. For example, vector database, cache system, labeling system, etc. These supporting tools and systems are crucial to the continuous iteration and development of products. During the research and development process, we need a complete tool chain and solutions to support data collection, preprocessing, labeling, and model training, optimization, and deployment. Mr. Chao Wang, do you have anything to add? ***
Wang Chao: In this question, I can share some information about the large model under development. We are developing our own industrial model and look forward to cooperating with enterprises and colleagues. More information on partnership opportunities will be announced after July. Also, mention how to verify which large models are better. In this regard, we are more concerned about how to successfully apply proven large-scale models on the platform, such as Baichuan and other models, and encourage everyone to pay attention to and understand them.
***Xu Wenhao: In terms of R&D efficiency, what experience do you have to share in terms of team size, computing power input, and time estimation for developing and deploying large-scale models? ***
Wang Suwen: The entire process of developing and deploying large models does take a certain amount of time. Especially in the compression, acceleration, and optimization of models, iterative experimentation and tuning are required, which can be very time-consuming. For example, we once developed a model with a scale of 7B. After optimization, on the 4-card b-type configuration on the A800 server, the inference speed was reduced from the previous 28 milliseconds to about 5 milliseconds. Total human effort depends on the work you do.
First, the entire basic framework needs to be constructed, and the model should be compressed and quantized, including operator optimization. We optimized based on Nvidia’s FastarTransformer, so we need to customize the optimization operator, choose an inference engine that suits the needs, such as Nvidia’s Triton, and provide services according to different backends. Finally, overall performance testing is required to determine the best performance of the model on different devices and for final deployment. According to our experience, the whole adaptation process takes at least one month. In addition, it will take some time to optimize the task of adjusting instructions, which depends on the specific business needs and the number of instructions. Depending on the type of business, a team of more than a dozen people is usually required to do it.
***Xu Wenhao: It sounds like it will take a team of about ten people, and it will take a month or two to complete the reasoning optimization and training process of these models. This is not the kind of large model with hundreds, thousands, or hundreds of millions of parameters we are talking about. ***
Wang Suwen: Yes, a well-trained and optimized model in a specific industry can be quickly deployed and replicated to customers in that industry. In this way, we are able to leverage our previous work and provide our clients with customized solutions. For example, we have optimized models in the financial field, insurance, wealth management and retail industries, so we can quickly replicate these optimized models and provide services to customers quickly. This ability to reuse can greatly improve efficiency and speed up the delivery of solutions.
***Xu Wenhao: I understand. In fact, it is just a process of product development at present, not a process of project landing. A product has been developed that can be used by many, many customers. What is Teacher Jia’s opinion on this issue? ***
Jia Haowen: For the development of large domain models, data collection is essential for domain-specific models. For highly specialized fields (such as law), data collection may take a long time, maybe half a month or even a month. After the data collection and structured processing are completed, the following links can start, such as the tuning of the instruction set and the training process of multi-level and multi-card. After completing these steps, multiple rounds of model effect evaluation are usually performed, because the results generated by the Transformer mechanism may not be reliable enough, and a large number of effect tests are required to ensure the reliability of the model. When the model training is basically completed and ready for initial commercial use, we may carry out further productization work according to the needs of customers, and package it into a complete product to provide users with comprehensive services.
From a cost point of view, data collation may take half a month to a month to train a relatively small model with a scale of 6B or 7B. However, this also requires an important prerequisite, that is, the team responsible for model training must be familiar with multi-machine multi-card training methods and methods, and be familiar with various data acceleration strategies and memory acceleration strategies. In addition, the preparation of the environment is also critical. For some small companies, if they want to train large models, they may need to rent machines on platforms such as Alibaba Cloud or Tencent Cloud and build their own environment. This additional cost also needs to be taken into consideration.
Develop a large model, what is the input-output ratio?
***Xu Wenhao: Based on the views of the two teachers, if the team conditions are mature, the development of a 7B-scale miniature model may require at least a team of more than ten people, and the development will take about three months. In this way, even developing a small model requires considerable investment. What is the approximate input-output ratio of developing a large model? How to solve regulatory risk and safety issues? ***
Jia Haowen: To evaluate the input-output ratio, specific business scenarios and needs need to be considered. For traditional copywriting, advertising, and film and television industries such as creative script generation, AIGC and other large models can quickly generate a large amount of material. Although the reliability may be lacking, it can significantly improve production efficiency. For these creative production tasks, the input The output ratio can be very cost-effective. However, for other fields such as legal knowledge generation, case analysis, insurance policy analysis, etc., because the output results may not be perfect, it needs to invest a lot of human resources and go through multiple rounds of model tuning to achieve a more ideal output. Therefore, the input-output ratio may be higher.
As for supervision, we also need to consider the recently introduced regulatory policies related to deep generation. It mainly involves several aspects. First of all, we need to pay attention to whether it will infringe the intellectual property or copyright of composers, writers, painters, etc., and whether it is easy to generate false information. In terms of regulation of generated results, we need to ensure supporting content review and management mechanisms to ensure that the generated text and images do not infringe intellectual property rights. At the same time, for traditional industries, compliance and risk control are also important considerations. As experiments develop, a large-model industry chain may form, with some working on means of production for large-model generation and others working on measures to prevent runaway large-model generation. The establishment of this correspondence will be iterated over time in order to achieve a good balance between the generation and supervision of large models under the framework of laws, regulations and ethics.
***Xu Wenhao: We will not only have AI companies, but also AI security companies, just like there are many security companies on the Internet. I would like to ask Teacher Wang Suwen’s views on AI safety and supervision issues. ***
Wang Suwen: First of all, when we are training domain models or large models, the compliance and legality of data are very important. We should obtain data through formal channels and ensure the security and confidentiality of data.
Secondly, when we conduct domain model training for customers, we must ensure the security and compliance of data within the enterprise, and the data between different enterprises should be isolated and cannot be reused or used in training at will. In addition, when training domain models for customers, we also need to perform fine-tuning and alignment to ensure that the output models meet compliance requirements.
From a regulatory perspective, government departments need interdisciplinary and cross-field expert knowledge when formulating regulatory frameworks. We cooperate with the Academy of Information Technology to jointly establish a sound regulatory framework and conduct in-depth cooperation with it. We are also conducting safety certification with the Academy of Science and Technology to ensure that our large models have been rigorously tested and verified.
Only through the cooperation of the government, experts, and enterprises can we promote the healthy development of AIGC, meet user needs, and ensure the safe use of large models.
In the field of intelligent customer service, what are the core barriers for enterprises?
***Xu Wenhao: In the field of intelligent customer service, everyone is doing multiple rounds of dialogue and sentiment analysis, so what are the core barriers of the enterprise? ***
Wang Chao: For the smart customer service industry, the homogenization problem is closely related to the goal of purchasing smart customer service and the expected ROI. In JD.com, when building smart customer service, enterprises should divide their development goals into three stages: primary, intermediate, and high (or mature). If the customer’s goal is still in its infancy, that is, only simple question and answer and inquiry functions, the needs of purchasing intelligent customer service may be similar. In this case, the degree of homogeneity may be higher. However, if the customer’s needs are positioned at a higher level, such as providing proactive services, full follow-up, and full-scenario services, then it is necessary to consider whether the smart customer service provider has a matching operating methodology and related complete operating tool system . Based on this logic, I think that one of the core barriers of the current generation of intelligent customer service products is whether the supplier has complex and mature robot project experience and rich operation experience.
Another key point is whether a smart customer service company can provide long-term planning and guidance operating methodology, as well as related supporting operating tool systems, in addition to providing relevant products and technologies. At the same time, whether we can help customers build a talent echelon and provide training services. These factors are very important in the current generation of bots.
As for the future, large models will be an important barrier. For the product you mentioned, whether different types of large models can be effectively integrated with the original product capabilities will be a key factor. At the same time, the ability to develop large models will also become a unique competitive advantage.
***Xu Wenhao: I believe that every company will feel that it has unique advantages in smart customer service or similar products. Mr. Jia, as far as your company’s products are concerned, where are its barriers? ***
**Jia Haowen:**From traditional customer service to large-scale model customer service, the whole process can be regarded as a barrier to competition. Although we may not be able to compete with big companies in terms of computing power and data volume, we may be able to compete in interdisciplinary fields, such as the application of psychological knowledge in large-scale model training, and the intersection of cognitive intelligence and large-scale model applications. Have a first-mover advantage. For other companies, they can also combine their own characteristics to stand out in the increasingly homogeneous large model service and training process.
Wang Suwen: This problem actually boils down to two core points: Smart customer service companies need to consider how to make profits and increase gross profit margins. In order to achieve this, there are two aspects to focus on first. First, you need to provide high-quality intelligent customer service to satisfy customers, so that your business can develop for a long time. Therefore, it is very important to pay attention to the effect of the product, including the improvement of product power and intelligent effect, so as to improve user experience and satisfaction. Second, focus on improving efficiency, consider the issue of input-output ratio, reduce costs and increase the gross profit margin of the project. The improvement of project delivery and operational efficiency is the key, and product satisfaction, deployment and implementation efficiency, and rapid integration with customer business systems and docking of operational content must be considered. You need to have a complete delivery methodology and operational tools to improve the gross profit margin of the project in order to achieve profitability and maintain sustainable development.
Smart customer service companies can be divided into two categories, one is professional manufacturers in vertical fields, and the other is general-purpose manufacturers. Smart customer service vendors in vertical fields focus on specific fields, such as e-commerce or insurance. Their advantages and barriers lie in industry focus, continuous optimization of industry knowledge maps and data, and provision of special solutions and core competitiveness. Zhongguancun Kejin is a conversational AI solution provider. We focus on finance, government affairs, retail and other industries. We have provided services to more than 900 industry leaders and accumulated rich industry knowledge. We also plan to launch general-purpose and large-scale domain models, and upgrade products such as intelligent customer service, outbound call robots, sparring and quality inspection assistants by integrating dialogue engines to enhance our competitiveness in the industry.
Second, improving delivery and operational efficiency is also key. Product satisfaction is crucial to reducing project delivery costs, and efficient deployment and implementation, as well as rapid integration with customer business systems and operational content docking, will improve operational efficiency. You need to have a set of delivery methodology and operational tools to ensure the maximum gross profit margin of the project. Doing so will allow you to be profitable and sustainable in the long run.
*Will AIGC completely replace traditional customer service staff? *
***Xu Wenhao: The three teachers mentioned three core barriers: the first is to focus on vertical fields, the second is to seek differentiation at the product level, and the third is interdisciplinary design. These measures will help businesses stand out in a highly competitive market and provide unique value to customers. So the last question for today, please use simple language to imagine the future development of AIGC in this field. Will AIGC completely replace traditional customer service personnel? ***
Wang Chao: As a practitioner, I am optimistic about the prospects of AIGC, and the issue of replacement involves different perspectives. One point of view is from the perspective of the stock market, and believes that the market space of the customer service industry is limited, so AIGC may replace traditional manpower. However, I prefer to think of it from an incremental perspective.
First of all, intelligent customer service still needs human operation support, and operators still play an important role in the transformation process from traditional customer service to intelligent customer service. Secondly, the future operation mode may change, and the cooperation between intelligent customer service and manual operation personnel will form a new operation mode. In this model, a small number of operators can carry intelligent customer service robots to provide high-quality 24-hour services at a lower cost, which will allow more small and micro enterprises to provide customer service in new ways and expand the market size. In short, from an incremental point of view, smart customer service will not completely replace traditional customer service, but complement it, bringing new opportunities and room for development to the market.
Wang Suwen: In the foreseeable future, human customer service will not be completely replaced, because they have unique advantages in dealing with complex, thinking and emotional issues. Especially in dealing with high-value customers, potential customers and improving customer conversion rates, human customer service still plays an important role. Due to the high cost of acquiring customers, it is still the hope of many companies to efficiently follow up and ensure the transaction through human customer service. Therefore, the relationship between human customer service and intelligent customer service is more of a cooperation model that combines each other. Enterprises need to consider the advantages of human customer service and intelligent customer service according to their own conditions, and formulate the best customer service model.
On the whole, I think the future development space of AIGC is broad, and the whole industry has also seen this. In the next two to three years, AIGC and technologies like ChatGPT will develop at a high speed and promote the upgrading of the entire enterprise service industry. Both the Internet and enterprise service industries will undergo large-scale upgrades and changes, including the improvement of supporting facilities. At present, there are still some problems in AIGC, such as content quality, investment cost, data security and copyright. Therefore, we still need more long-term development, including exploring more sophisticated and effective modeling methods, to improve these problems. I believe that with the advancement of technology, the development space of large models will be infinitely broad.
**Jia Haowen: **Indeed, we should not place too much emphasis on substitution, but focus on the changes in working and business models that will be brought about in the future. In the process of promoting business, we need to measure the ratio of input to output, especially in the promotion of customer service, we need to consider the security of user data privacy, compliance with laws and regulations, and cross-scenario and cross-industry anthropomorphic services. Large models can bring high value to traditional customer service personnel. They will bring about qualitative changes, but they do not mean replacing human customer service. Overall, although the large model has some problems at present, it has great prospects in the future. To put it in a more literary way, the development of large models will change from a dream to a reality in the near future, and we will soon be able to experience it ourselves.
