Creating an Effective Prompt for the PMTA Chatbot

May 21, 2024

I sometimes get questions from users of the PMTA chatbot I built and continue to train with new information about how to best extract information from the tool. I call it the "blank paper syndrome"- when you're sitting in front of a blank screen and you don't even know how to start articulating the ideas you have in your mind. The art and science of chatbot prompting is similar in many ways to that problem. So naturally I wrote an SOP to help users get creative and understand how the model can be of service to the work in the nicotine and tobacco regulatory space.

Standard Operating Procedure (SOP) for Creating Effective Prompts for the PMTA Chatbot

1. Purpose The purpose of this SOP is to provide guidelines for creating effective prompts to interact with the PMTA chatbot. These prompts will help users obtain accurate and relevant information from the chatbot, which is trained on PMTA marketing granted orders and their respective TPLs.

2. Scope This SOP applies to all users who interact with the PMTA chatbot for information retrieval and analysis.

3. Responsibilities

• Users: Follow the guidelines outlined in this SOP to create effective prompts.

• Administrators: Ensure that users are aware of and understand this SOP.

4. Definitions

• Prompt: A question or statement inputted by the user to elicit a response from the chatbot.

• PMTA: Premarket Tobacco Product Application.

• TPL: Technical Project Lead.

5. Procedure

5.1. Be Specific and Clear

• Example: Instead of asking, "Tell me about PMTA," ask, "What are the key components of a PMTA application?"

5.2. Use Complete Sentences

• Example: Instead of typing, "Toxicology review," ask, "How does a toxicology reviewer assess a PMTA application?"

5.3. Focus on One Topic at a Time

• Example: Instead of asking, "What is PMTA and how is it reviewed?" ask, "What is the PMTA process?"

5.4. Avoid Ambiguity

• Example: Instead of asking, "What about marketing orders?" ask, "What information is included in a PMTA marketing granted order?"

5.5. Use Contextual Keywords

• Example: Include relevant terms like "substantial equivalence," "toxicology," or "social science" to guide the chatbot.

• Example: "Explain the social science review process in a PMTA application."

5.6. Ask Open-Ended Questions

• Example: Instead of asking, "Is toxicology important?" ask, "Why is toxicology important in a PMTA application?"

5.7. Be Concise

• Example: Instead of a long-winded question, ask, "What are the steps in the PMTA review process?"

5.8. Use Proper Grammar and Spelling

• Example: Ensure your prompt is free of typos and grammatical errors to avoid confusion.

5.9. Provide Context When Necessary

• Example: If your question is based on a specific document, mention it. "According to the PMTA Final Rule, what are the new requirements for PMTA?"

5.10. Test and Refine Prompts

• Example: If the initial response is not satisfactory, refine your prompt. "Can you provide more details on the toxicology assessment in PMTA?"