Historical Policy Analysis with ChatGPT

How to use ChatGPT to conduct policy analysis with an eye to historical patterns

Have you ever faced the challenge of conducting a detailed policy analysis on a topic that's new to you, or under a time crunch?

"Hey Zoe, I need a deep dive into the recent changes in marketing laws over the last decade. I'm trying to understand the journey we've taken to get here. Could you have that ready by the end of the day?"

If this request sounds overwhelming, you're not alone. Traditionally, this would involve a time-consuming search through countless government websites and academic papers, a task that could feel insurmountable within a single day. But, there's good news: ChatGPT can expedite things for us.

In this guide, I’ll walk you through how to leverage ChatGPT for policy analysis. Together, we'll explore how to:

• Identify Patterns and Trends: You'll learn how to have ChatGPT sift through historical policy documents and legislative changes.

• Comparative Analysis: We'll compare how similar policies have evolved in different regions or under varying administrations.

• Impact Assessment: Finally, we'll assess the outcomes of past policies.

The end result: a 30-page primer.

Let’s get to it!

The task

• We’ll examine the progression of environmental policies over the past decades to reveal trends in regulatory focus, legislative changes, and government action.

• Then we’ll compare how similar policies have evolved in different regions and administrations.

• We finish by evaluating the outcomes of past policies to understand their effectiveness and public reception.

Part 1: Set the frame and define the terms

To start, we need to set the frame for the prompt:

I'm a regulatory expert and you are my personal policy analyst. You are an expert in analyzing policy trends from a historical perspective, with an ability to assess how these trends may impact the future.

I write for an audience of corporate executives and need your help explaining specific policy trends, impact, and outcomes.

Your task: Analyze the evolution of renewable energy policies in Canada over the last 5 years.

You MUST identify key legislative milestones, notable political changes, and the impact of these policies on renewable energy adoption.

Your analysis MUST be thorough and factual

So here is what we're going to do to start:

- Break down the last 5 years into 3 sections. 
- In each of these sections, provide a 250-350 word summary of how the legislative, political, and opinion environment evolved.

I will then provide you further instruction on how to expand each section.

Please begin.

There are a few things going on here:

• Assigning it a role. “I'm a regulatory expert and you are my personal policy analyst. You are an expert in analyzing policy trends from a historical perspective, with an ability to assess how these trends may impact the future.” You'll see me do this often in future guides. I find I get higher quality responses when I give ChatGPT a specific role to play.

• Provide context. We share details of what we need ChatGPT’s help with in its new role. We then outline our target audience for the analysis and the themes we want to capture.

• Give it clear rules. The part where we say “you must” is the result of many failed attempts to get this from ChatGPT in my original prompts. I had to try multiple iterations and specific follow-up with ChatGPT to extract these answers. This ensures we get those answers on the first go.

• Preventing a Load Error. My original prompt was long and detailed. It appears, too long and too detailed. ChatGPT would always experience a load error as it began writing its response. To prevent this, I had to guide it through the analysis, bit by bit. This is why I only ask for a summary of each section.

Here is the output I received from ChatGPT:

Part 2: Dig Deeper

From here, we want to break down each section to dig deeper.

I find it easiest to copy the initial response I get from ChatGPT to a new document so that I can copy and paste to it, while keeping myself organized. Then:

• For each section, I ask a series of pointed follow-ups and specifics on the type of analysis I’m looking for:

Let's start with the first section, {{insert section title}}.

- Expand on this to include a 250-400-word analysis of the how the regulatory and legislative environment changed, and the impact these changes have had on today's regulatory landscape.

Once you have moved the answer to your main document, proceed to the next level of analysis. With this output, you could choose to replace the original section with this content, or add to what you have and make some edits. Here is the output I got:

You can see where I’m going with this now. I’m going to build on this with my next deep-level prompt:

For this period, identify key stakeholders and other groups that drove these changes, and the public arguments they used to advance their agenda. Likewise, note stakeholders that were opposed, and their rationale for their opposition.

Here is what I want to avoid:
- Generalizations. Do not give me generic and broad answers. I need specific references to political parties, political leaders, and stakeholders. For example, it's not sufficient to make note of oppoisition from the "oil and gas sector." I need to know which companies specifically opposed.

I had to add the rule in this prompt to increase the quality of the output, as the first iterations were way too vague and predictable. As you see with the output, I got higher quality analysis.

Let’s proceed:

For this period, outline the political conditions that motivated these changes and what messaging was used by the various actors.

I’ll spare you a wall of additional screenshots — I’ve included a Google Doc of the final output I got by following all the prompts below, but as you see, I focus on digging deep on the types of topics I always want to see in a policy analysis.

For this period, how did public opinion polling evolve in this time?

Interestingly, with ChatGPT’s ability to search the Internet, it may include specific links for each of its bullets with this prompt. However, it is not always consistent in doing this, so if you want reference material, you’ll need to ask for it.

For this period, identify patterns and trends, citing specific policies.

Then, take a look at how different governments have tackled this:

For this period, compare how similar policies have evolved in different regions or under different administrations within Canada.

And finally, we pull out an impact assessment.

For this period, evaluate the outcomes of past policies to understand their effectiveness and public reception, which often guides future policy directions.

Repeat this for all three sections, and you’re left with a robust analysis in raw form that you can edit to suite your specific needs.

Let's move on to the next period, {{insert copy that ChatGPT originally provided for your second section.}.

Your Task:

Expand on this to include a 250-400-word analysis of the how the regulatory and legislative environment changed, and the impact these changes have had on today's regulatory landscape.

And then work your way through each sub-section as you did before.

Additional Notes

• I have experienced error messages in the past where ChatGPT tells me the conversation is too long. You may want to start a new chat for each major section of your analysis to prevent this issue, using the original response you get from the first prompt to kick off the workflow above.

• I also experienced error messages when I originally structured my prompt to insist that every answer include links to reference material. Removing this requirement allowed me to proceed as you’ve seen above, but I would conduct a rigorous fact-check before finalizing this analysis.

• You’ll see that the raw export I have from these prompts is large. It’s a 30-page document, and on its own, it’s not one I would share with colleagues or clients. However, it serves as a robust basis from which to edit.

• You could also upload this raw export to ChatGPT and ask it to write an executive summary for it, or make further edits.

That’s it!

I hope you enjoyed this guide and that it helps you do amazing public affairs work. Let me know what you think in the comments!

Chat soon,

Joseph