The dreaded media analysis. You know what I’m talking about. Hours spent gathering data, coding it, analyzing it, trying to make sense of it, and then explaining to an executive audience why any or all of it matters. It can be daunting.
But it’s less daunting with ChatGPT.
Today, we’ll explore how to:
Prompt ChatGPT to provide an initial analysis of data, highlighting major themes and insights.
Understand how to refine ChatGPT's sentiment analysis for more precise and contextual insights.
Find out how to leverage ChatGPT's analytical capabilities to draft strategic considerations and actionable plans.
Keep in mind that ChatGPT's analysis is limited to the data it's given and the context it understands. Therefore, the accuracy and usefulness of its insights depend on the quality and representativeness of the data. In other words, “Garbage in, garbage out.”
With that caveat, let’s put it to use. You can also skip ahead to the prompt index.
Step 1: Data Collection
For your own analysis:
Collect Data: Gather a substantial dataset of text related to the public affairs issue you are interested in. This could include social media posts, forum discussions, comments on news articles, and public opinion pieces.
Organize Data: I’ll do a future post on data cleaning and analysis. For now. structure the data in a way that is easily processable. This might mean categorizing data by source, date, or other relevant factors.
For this guide, I wanted to explore how the November 2023 drama around OpenAI shaped media coverage, and sentiment towards OpenAI, as well as Sam Altman, and the Board.
To get a basic data source, I searched for the topic, “OpenAI” on Google News and then clicked on the “Full Coverage” button for each sub-topic.
For each of those, I used Instant Scraper, a Chrome Extension, to download headlines and sub-headlines. Obviously a better data source would be a full export of headlines and body from an enterprise-grade monitoring platform like Brandwatch or Meltwater, but to keep things simple today, I’ll stick with basic headline analysis.
Once I scraped and cleaned my data, I ended up with 222 headlines headlines. This took me 15 minutes to compile. You can access the sample data here.
Step 2: Initial Analysis
To start, I wanted to see what type of baseline analysis ChatGPT could offer. I uploaded my file to ChatGPT with this prompt:
You are a data analyst with 20 years of qualitative reasearch experience. You're able to analyze news headlines to surface themes, insights, interesting angles, and impact on a brand's reputation.
In the attached file, I have recent headlines for OpenAI, a technology company.
To start, please analyze the file and give me a topic analysis to surface major themes.
Here is what I got:
Not bad. This gives me confidence that ChatGPT has a good grasp of the source content. Let’s dig deeper.
Step 3: Brand Mentions
Next up, I wanted to get a snapshot of the brand and individuals that got the most mentions in these headlines. Rather than get a ranked list, I thought a word cloud would be more useful:
Please analyze these headlines and provide me a word cloud of the most frequently mentioned brands and individuals.
Here’s what I got:
Step 4: Publications and Reporters
I was curious to see which news outlets and reporters gave the most “ink” to the story:
Please analyze the file and list the top 5 sources that covered this story. The sources are labeled in the column called "Source"
Well, that saved me a lot of manual work:
Using the same spreadsheet from my previous request, please analyze the file and list the top 5 reporters that covered this story. The reporters are labeled in the column called "Authors". Be aware that some articles are written by multiple authors. Be sure to treat them all individually
Interestingly, my original data source was faulty. Some of the rows had the dates and authors in the wrong columns as a result of how the scraper treated the data. ChatGPT was able to identify this problem and noted it for me. What you see below is based on my corrected spreadsheet, but I was impressed that it could detect the inconsistent data.
Now I decided to put ChatGPTs skills to work. You’ll note in the data file, I have no biographical detail about any of these journalists. Here’s what ChatGPT could produce on its own:
Can you provide me with a biography of each of the 5 reporters you identified above? Limit each to 300 words or less, and be factual. I'm interested in their professional background and the topics and industries they tend to cover.
The results are not perfect:
Rachel Metz no longer works at CNN. She now works at Bloomberg, as noted in the source data, and with a quick Google Search.
Dave Paresh writes for WIRED now (but formerly wrote for Reuters).
The lesson? Before I use any of this information, I’d conduct a fact check, but for the most part, the contextual information about the topics and industries covered is strong.
Step 5: Sentiment Analysis
Let’s turn to a simple sentiment analysis. This is a qualitative exercise. What’s considered “positive” is entirely dependent on who you’re conducting the analysis for, or what side of an issue you’re on. Sentiment analysis from this perspective requires multiple takes. For this demo, I broke down the analysis for these entities:
OpenAI
Sam Altman
OpenAI’s Board.
I began by seeing what ChatGPT could do on its own:
Using my spreadsheet, provide me with a detailed sentiment analysis by entity. Let's start with OpenAI. Identify which posts are positive, negative, or neutral towards OpenAI. Quantify each of these categories using percentages
Massive turmoil in the executive ranks is hardly a positive story. I was curious about the relatively high ratio of positive stories, and asked ChatGPT to give me examples of headlines and how it categorized each by sentiment. Here is what I got:
Please provide me 5 examples for each of these categories
Not bad, but I wanted more precision to reflect the fact that executive turmoil, in general, is bad for OpenAI:
As a rule, any corporation that is experiencing turmoil in its executive ranks, is dealing with a reputational challenge. This headline, "Altman Is Back at OpenAI, But Questions Remain as to Why He Was Fired in First Place" is negative for OpenAI as it highlights the degree to which OpenAI may have created the conditions for this mess. Likewise, this is not a very positive headline for OpenAI: "Unresolved leadership crisis at OpenAI: Investors, employees, and legal issues in focus".
Given this perspective, can you please refresh your sentiment analysis?
With this correction, ChatGPT adjusted its initial analysis:
This makes more sense. From OpenAI’s perspective, there was absolutely nothing positive about this story. I could fine-tune this analysis even more, but let’s move on.
What about Sam Altman?
Let's move on to Sam Altman. Using my most recent spreadsheet, identify which stories are positive, negative, or neutral towards Sam Altman. Quantify each of these categories using percentages.
I repeated the steps from above, asking for 5 headlines from each category and then providing ChatGPT with additional context before asking for a fresh cut of the numbers.
Step 6: Identifying Attitudes, Concerns, and Interests
Beyond these numbers and baseline sentiment analysis, I wanted to see how good ChatGPT would be at zooming out and looking at the big picture.
What are the main concerns expressed in negative posts about OpenAI?
Pretty darn good. I could repeat this prompt for each of the entities and brands I’m interested in, by sentiment and have a robust view of the themes driving media coverage.
Step 7: Initial Summary
I covered a lot with ChatGPT, so I thought it might be helpful to get one summary of the work we did on sentiment analysis to avoid load errors or hallucinations later:
Can you summarize all of your findings from above into a report format?
Step 8: Strategic Considerations
Given the analysis above, I wanted to see how well ChatGPT could contextualize media coverage for an executive audience. With this simple prompt, I got a decent answer:
What are 3 major strategic considerations OpenAI should consider, given this media coverage?
Step 9: Action Plan
And finally, to turn this analysis into actionable insight, I asked ChatGPT to produce a 30-day action plan to deliver on its assessment above.
Can you provide me with a 30-day action plan that would address these considerations. Please account for internal communications, external communications, stakeholder engagement, and the general public. Keep the plan focused on concrete, measurable, and achievable actions.
Is this the plan I’d produce? Probably not. But it gives me something concrete to work from. And that’s a big time saver:
Step 10: Wrap it up with an executive summary
By this point, I’ve had a lengthy exchange with ChatGPT, going into the weeds on numbers, topics, sentiment, and considerations. I wanted to zoom out completely and asked for an executive summary.
Can you produce an executive summary of 500 words or less that outlines the results of the media analysis you conducted above, what it means, and what comes next?
Additional Notes
These prompts and results are based on a basic spreadsheet of media headlines. If I was conducting a detailed media analysis, I’d do it using the full text of each article. However, doing so would likely cause performance issues with ChatGPT, so I’d be prepared to break it down into sections to avoid load errors.
Be prepared to refine your queries and train ChatGPT as you conduct the analysis. This is pretty much what you’d need to do if you were working with an analyst, so it should go without saying, but in my experience, the more you train ChatGPT, the better it performs.
That’s it!
I hope you enjoyed this guide. Let me know what you think in the comments!
Chat soon,
Joseph
Just the Prompts
Initial Analysis:
You are a data analyst with 20 years of qualitative reasearch experience. You're able to analyze news headlines to surface themes, insights, interesting angles, and impact on a brand's reputation.
In the attached file, I have recent headlines for OpenAI, a technology company.
To start, please analyze the file and give me a topic analysis to surface major themes
Brand Mentions:
Please analyze these headlines and provide me a word cloud of the most frequently mentioned brands and individuals.
Publication Analysis:
Please analyze the file and list the top 5 sources that covered this story. The sources are labeled in the column called "Source"
Reporter Anaysis:
Using the same spreadsheet from my previous request, please analyze the file and list the top 5 reporters that covered this story. The reporters are labeled in the column called "Authors". Be aware that some articles are written by multiple authors. Be sure to treat them all individually
Then:
Can you provide me with a biography of each of the 5 reporters you identified above? Limit each to 300 words or less, and be factual. I'm interested in their professional background and the topics and industries they tend to cover.
Sentiment Analysis:
Using my spreadsheet, provide me with a detailed sentiment analysis by entity. Let's start with OpenAI. Identify which posts are positive, negative, or neutral towards OpenAI. Quantify each of these categories using percentages
Sentiment Calibration:
Please provide me 5 examples for each of these categories
Sentiment Correction:
As a rule, any corporation that is experiencing turmoil in its executive ranks, is dealing with a repetitional challenge. This headline, "Altman Is Back at OpenAI, But Questions Remain as to Why He Was Fired in First Place" is negative for OpenAI as it highlights the degree to which OpenAI may have created the conditions for this mess. Likewise, this is not a very positive headline for OpenAI: "Unresolved leadership crisis at OpenAI: Investors, employees, and legal issues in focus".
Given this perspective, can you please refresh your sentiment analysis?
Attitudes, Concerns, Interests:
What are the main concerns expressed in negative posts about OpenAI?
Summary of baseline analysis:
Can you summarize all of your findings from above into a report format?
Strategic Considerations:
What are 3 major strategic considerations OpenAI should consider, given this media coverage?
Action Plan:
Can you provide me with a 30-day action plan that would address these considerations. Please account for internal communications, external communications, stakeholder engagement, and the general public. Keep the plan focused on concrete, measurable, and achievable actions.
Executive Summary:
Can you produce an executive summary of 500 words or less that outlines the results of the media analysis you conducted above, what it means, and what comes next?