International Research Journal of Engineering and Technology (IRJET)
e-ISSN: 2395-0056
Volume: 11 Issue: 08 | Aug 2024
p-ISSN: 2395-0072
www.irjet.net
Efficient Prompt Design Automation for Large Language Models by Parts of Speech tagging leveraging Viterbi Algorithm Sayan Guha1 Associate Director & Principal Architect, AI & Analytics Practice, Cognizant Technology Solutions, West Bengal, India ---------------------------------------------------------------------***--------------------------------------------------------------------1.1 Motivation Abstract – Maintaining a standard in writing prompts to interact with Language models requires a disciplined learning and approach on the acceptable precision, conciseness and brevity of the prompts which effectively produces results better in cases where such standards are maintained as compared to the situations where it is not maintained. The prompt engineering practice guidelines states to have only actions which relevant and which can produce precise results as compared to incidents where standards are not maintained. The manual process of prompt design could be accelerated by a system which includes Viterbi algorithm-based Path Pruning which would accept raw English language input and would prune the path which is most relevant and would discard the paths which have lower probabilities. In this process, the Viterbi algorithm would identify the key elements in Parts of Speech tagging and will align and assemble the natural language to fit a prompt template in the order expected by the Large Language model and in the right practice of prompt design.
My motivation for writing this paper is to accelerate and simplify the process of prompt engineering, making it accessible to a broader audience, including those who may not be skilled in writing prompts according to standard practices. By leveraging the Viterbi algorithm, I aim to transform raw, unstructured inputs into effective and suitable prompts. This approach not only streamlines the prompt design process but also ensures that high-quality prompts can be generated efficiently. The system I propose will take raw prompts as input, apply the Viterbi algorithm to identify the best possible prompt, and discard less-effective options through path pruning. This method will facilitate an automated, time-efficient solution, empowering individuals to create effective prompts without needing extensive expertise in prompt engineering. Ultimately, this research seeks to scale the practice of prompt engineering, making it more inclusive and efficient.
1.2 Aim of this paper
Key Words: Prompt Engineering, Viterbi Algorithm, Path Pruning, Prompt design, Few Shot prompting, Natural Language, Parts of Speech (POS) Tagging, RTF (Role Task Format), Generative AI, Large Language Model (LLM), Artificial Neural Network (ANN), Hidden Markov Model (HMM)
The aim of this paper is to develop a novel approach to efficient crafting of prompts that accelerates and simplifies the process, making it accessible to a broader audience, including those without specialized skills in writing prompts according to standard practices. By leveraging the Viterbi algorithm, this research seeks to transform raw, unstructured inputs into effective and suitable prompts. The proposed system will take raw prompts as input, apply the Viterbi algorithm to identify the best possible prompt, and discard less-effective options through path pruning. This method aims to streamline the prompt design process, ensuring the efficient generation of high-quality prompts. Ultimately, this research endeavors to scale the practice of prompt engineering, making it more inclusive and efficient, thereby empowering individuals to create effective prompts without needing extensive expertise in the field.
1.INTRODUCTION In today’s rapidly evolving technological landscape around prompt engineering practice evolved as a major component for Generative-AI driven solutions the ability to design effective prompts remains a critical challenge, especially when dealing with raw, unstructured English inputs. To address this, I propose leveraging the Viterbi algorithm to identify the optimal prompt while discarding low effective options through path pruning. This approach aims to automate and streamline the prompt design process, making it more accessible and time-efficient for individuals who may lack expertise in standard prompt writing techniques. In this paper, I will explore how this method can facilitate the creation of high-quality prompts, thereby enhancing productivity and ensuring consistency in various applications.
© 2024, IRJET
|
Impact Factor value: 8.315
2. LITERATURE REVIEW A literature review was undertaken encompassing few academic and industry papers. This section reviews Prompt engineering best practices as undertaken by organizations & the key aspect of prompt design can be achieved using Viterbi algorithm.
|
ISO 9001:2008 Certified Journal
|
Page 656