International Research Journal of Engineering and Technology (IRJET) e-ISSN: 2395-0056 Volume: 11 Issue: 07 | July 2024
www.irjet.net
p-ISSN: 2395-0072
Patent Analysis on Generative Language Models Sheela N S1, Dr. J Venkata Krishna2 1 Research Scholar, Dept. of Computer Science and Engineering, Srinivas University, Mangaluru, Karnataka, India 2 Associate Professor, Dept. of Computer Science and Engineering, Srinivas University, Mangaluru, Karnataka,
India ---------------------------------------------------------------------***---------------------------------------------------------------------
Abstract - Nowadays the evolution of Artificial Intelligence
various data sources including books, internet, articles and other.
is covering the large area of applications. The Generative AI models gaining more popularity as they are capable of generating the new content as humans using Machine Learning and Neural Networks. In this patent analysis paper, we are collecting the patents related to Generative Language Models and analyzing them in terms of number of patents per year, assignee and geographical area. This growth of number of patents in Generative Language Models provides us the way for future research work.
A deep learning neural network is used to make the model to learn patterns and relationships in the text to mimic human. The neural network architecture used is the transformer architecture. This architecture is based on self-attention mechanism the essential one to understand the language [3]. Tokens are the smaller units of text used by the language models to process the text. The statistical relationship between the tokens are analysed and the next token is predicted in the text sequence. Tokenization varies from model to model.
Keywords: Artificial Intelligence, Generative AI, Generative Language Models, Machine Learning and Neural Networks
Reinforcement Learning from Human Feedback(RLHF) is used to train the language models. RLHF is a technique which allow the language model to align to human preferences. It has been evolved from Preference-based Reinforcement Learning (PbRL) and has wide area of application including natural language processing and computer vision [4].
1.INTRODUCTION Deep learning is a subset of artificial intelligence that is based on the neural networks. The neural networks can be trained using supervised, unsupervised and semi-supervised algorithms [1]. Deep learning has wide area of applications including computer vision, natural language processing, medical image processing. Generative AI (GenAI) is a type of AI technology capable of generating new data like text, image, video and other based on the data it is trained. GenAI is the subset of deep learning, uses artificial neural networks. It can process labeled and unlabeled data using supervised, unsupervised and semi-supervised models. It is capable of generating new data like text, image, video and other based on the data it is trained.
2.2. Applications Generative language models are evolving randomly and have significant role in various domain. Text generation- Language models can generate coherent, contextually relevant text and controllable text. A transformer-based pre-trained language model is used with some modification to generate controllable text [5, 6, 7].
2 GENERATIVE LANGUAGE MODEL
Language Translation- Language models are used to translate text from one language to another.
Generative language model is an Artificial Intelligence used to generate the text that are relevant to the context based on the prompt it received. This model is trained on vast amount of text data in order to understand and interact in the way the humans are. Some of the examples of Generative Language Models are GPT (Generative Pre-trained Transformer), BERT (Bidirectional Encoder Representations from Trnasformers), T5 (Text-To-Text Transfer Trnasformer), XLNet, OpenAI Codex.
Content creation- Generative Language Model can be used to create the new contents including text, audio, video and it is often cannot be distinguished from human content [8]. Conversational AI– generative AI models like ChatGPT, chatbots and virtual assistants are used in variety of tasks during conversation like generating text, summarizing text and answering questions.
2.1 Working
3. PATENT ANALYSIS
Generative language models are trained on large training datasets which are both labeled and unlabeled [2]. The training datasets is large and comprising of diverse text from
© 2024, IRJET
|
Impact Factor value: 8.226
A patent is an intellectual property right, to protect inventions and gives legal right to the owner for limited
|
ISO 9001:2008 Certified Journal
|
Page 1331