Finished course on Coursera “ChatGPT Advanced Data Analysis” by Vanderbilt University. Tried to answer the question “Is this a good problem for ChatGPT Advanced Data Analysis?” in the post. Also added useful notes, tips & tricks, you’d need to keep in mind while prompt engineering, which I call “best practices”

Advanced Data Analysis is available only with a ChatGPT+ or Enterprise subscription. To enable Advanced Data Analysis, you will need to go into the “Settings & Beta” in ChatGPT, then enable “Advanced Data Analysis” (the tool formerly known as Code Interpreter)

When working with Code Interpreter, think of it as your intern or assistant.
You will need to:

• Break things down step by step
• Provide all the necessary context or information to accomplish a task
• Provide detailed feedback on what to improve
• Do not expect the work to be perfect or error-free
• Always review everything for accuracy
• Don’t expect Code Interpreter to be a mind reader, you need to be very specific about your goals, constraints, needs, etc.

Is this a good problem for ChatGPT Advanced Data Analysis?

How to determine if we should be trying to solve a given problem with ChatGPT Advanced Data Analysis? Some fundamental considerations below.

1. It’s easy to check if the solution is correct and errors won’t create new problems.

ChatGPT Advanced Data Analysis can and will make mistakes. Even though ChatGPT Advanced Data Analysis creates Python code to perform many operations, it can still make mistakes in the Python code that it produces. You should focus on problems where it is OK to make errors or where errors are easily detected.

For example, an application to help you brainstorm hard questions that might be asked about your presentation is very error-tolerant. Generating lousy questions or not thinking of good questions won’t cause harm — it just may not be helpful. An application to determine whether or not a human gets a loan is a poor use case as a mistake in the solution could cause great harm.

The solution must be much more time consuming to find than it is to check for correctness. It’s easy to check if the solution to a crossword puzzle is correct, but its much harder to think up solutions. It’s much easier to check if a meal plan meets your desired tastes than it is to think up the meal plan.

2. A partial or flawed solution to the problem has value.

Problems where a partial or flawed solution still has value align well with the ChatGPT Advanced Data Analysis due to its ability to facilitate iterative development, debugging, and rapid prototyping. The interactive nature of the platform enables users to explore different approaches, make incremental adjustments, and learn from mistakes, all of which can lead to a better understanding of the problem and more robust solutions. Even if a solution isn’t perfect, it can be valuable for specific purposes, constraints, or educational goals.

ChatGPT Advanced Data Analysis’ flexibility and adaptability allow users to focus on key aspects of a problem and provide solutions that meet critical requirements without necessarily achieving perfection. This balance between perfection and practicality optimizes the use of resources, making it a suitable tool for a wide range of scenarios, including those where rapid development or resource optimization is essential.

What are examples of problems where a partial solution has value? Any area where you need additional insight into a problem, such as creating questions related to a document, helping spot conflicts in decisions made in different meetings, providing a second check that didn’t previously exist on a human decision, generating sample marketing messages, outlining a complex document, or identifying potentially relevant information are all cases where a partial solution can have value.

3. The solution to the problem will support your creativity and thought.

ChatGPT Advanced Data Analysis and other similar AI tools have amazing benefits, but only when they support your thought and creativity. Your thought provides the creative spark of the really great outputs from these tools. If the way you are using the tool will make you think less, it isn’t worth using the tool. Before you use ChatGPT Advanced Data Analysis for the task, ask yourself, does this really help me think and create more.

Supporting your creativity and thought can take a number of forms, ranging from automating tedious tasks to allow you more time to think to showing lots of ways of solving a problem to help you learn. If you are using ChatGPT Advanced Data Analysis to cheat on an exam and avoid learning something new, you aren’t using it correctly. If you are using ChatGPT Advanced Data Analysis to deepen you understanding of how to solve certain types of problems by seeing how it proposes to work through examples, while acknowledging that its approaches may be wrong, then you are using it correctly.

10 Best Practices for ChatGPT Advanced Data Analysis

1. Saving Intermediate Files

Why: To prevent loss of progress, allow assembly of multiple outputs, and enable various directions based on intermediate files.

How: Regularly save files at different stages of the project, allowing for flexibility in assembling outputs or taking the idea in multiple directions. These act as safety nets, ensuring that no work is lost. If ChatGPT Advanced Data Analysis times out, provide it the files that you previously downloaded. You can also ask for a Zip file with all the files generated thus far. You can also take an intermediate output and iterate on it in multiple independent conversations by uploading it to each one.

2. Always Plan Step-by-Step

Why: For a seamless process that enhances repeatability.

How: Collaborate with ChatGPT Advanced Data Analysis to outline specific, detailed, and actionable steps. Save these plans for future reference or to restart a process, making the process more reliable. You can use human or AI planning, but having a step by step plan is key. Save plans as a file so that they are easier to repeat by uploading them at the start of a new conversation.

3. Use Mementos

Why: To have reminders of the state of the ongoing process.

How: Have ChatGPT Advanced Data Analysis create files listing the entire plan, the current step, and summarizing what has or hasn’t been done. These mementos are different from the plan itself and act as a guide to the current state of the process. You can upload these to restart a process from a given point or help Code Interpreter remember where it is in the process.

4. Always Have ChatGPT Advanced Data Analysis Read and Explain Documents/Data

Why: To synchronize your understanding with GPT-4’s understanding.

How: Request ChatGPT Advanced Data Analysis to explain what it reads in several ways and test its understanding by having it generate examples or explanations that demonstrate the concepts in the document. Make sure that you and Code Interpreter have the same understanding regarding the documents, data, etc. that you provide. Asking it to generate new examples of concepts is a great way to test reasoning.

5. Use Error Detection Methods

Why: To ensure consistency and accuracy.

How: Employ references to specific identifiers or quotations to ensure that the output is supported by the documents you provided. Build custom test cases with real or synthetic data to ensure outputs are consistent with the input documents. If you can’t work with real data yet, generate synthetic data to test out a process and accuracy.

6. Ask ChatGPT Advanced Data Analysis to Try Alternate Approaches

Why: To overcome failure and make continuous progress.

How: If an approach fails, prompt ChatGPT Advanced Data Analysis to explore different strategies. Provide hints or ask it to plan, list, describe the rationale behind, and employ alternate methods.

7. Think of Prompts as Constraints

Why: For targeted and desired outputs.

How: Be explicit about your goals, requirements, and constraints. ChatGPT Advanced Data Analysis will try to generate any solution that fits, so the more specific you are, the closer the output will be to your needs. Don’t ask for a fruit if you really want a green banana.

8. Edit the Conversation When Errors Occur

Why: To avoid error propagation.

How: Edit and correct any chat message that produces a bad output immediately and regenerate the output. Clean conversation histories are preferred to prevent introducing erroneous information that causes problems later.

9. Get Key Information into the Conversation

Why: To enhance reasoning and understanding.

How: Include all necessary information directly in the conversation so you and ChatGPT Advanced Data Analysis have a common understanding. Visibility of information makes reasoning more effective. If you can see the information in a recent chat message, Code Interpreter can as well.

10. Tell ChatGPT Advanced Data Analysis to Analyze Without Python When Needed

Why: Not all tasks require code.

How: If you are getting poor reasoning on unstructured text, direct ChatGPT Advanced Data Analysis to analyze, read, or summarize “without using Python code” or “manually”. In most cases, you don’t want Python code doing the textual analysis.