Building an AI Powered Virtual Newsroom Deep Dive
Part II: Prompt Engineering for highest quality results
Perfecting Equilibrium Volume Four, Issue 3
This is a gang, and I'm in it
My man Dre will mess you up in a minute
With a right, left, right, left you're toothless
And then you say damn they ruthless!
The Sunday Reader, April 27, 2025
When I was a small boy I loved those cartoon Hawaiian Punch commercials where a crazy kid carrying a tray of Hawaiian Punch drinks walks up to an older man and says “How about a nice Hawaiian punch?” And when the old guy says Sure, the kid punches him straight in the kisser.
That’s always funny for a boy of 6.
Editor’s Note: That’s always funny for boys of 6 or 66 or any other age. Which explains the eternal popularity of the Three Stooges and the like.
Dealing with Large Language Models – so-called AIs – is best thought of as an exercise in dealing with that Hawaiian Punch kid: The more detailed and specific your instructions, the less likely you are to get punched straight in the kisser.
In Part One, we looked at the ideas and the concepts behind an AI-powered virtual newsroom, and how off-the-shelf Large Language Models can be tuned to give the best results.
Today in Part Two we're going to do a deep dive into the conclusion that we reached in Part One, which is that the key to this is going to be prompt engineering.
So we're going to take the prompts that we showed in Part One, feed them into a Large Language Model, and check out the results. Fortunately, the timing is good for us to try this out by doing some actual journalism. Texas has a legislature that meets every other year and passes a two-year budget. And they're in the process of passing the budget for 2026 and 2027 right now.
So let’s take the generic prompts we created last week, make them specific for the Texas budget, then feed them to Grok, which has lately been my Large Language Model of choice.
Let's ask Grok to take a look at the current budget and compare it to the new proposed budget, which is Journalism 101 for any story about a budget proposal. We’ll start with a very basic prompt, then see how Grok’s output changes as we make the prompts more and more specific and formatted.
We’ll start with this prompt:
Write a three-paragraph summary of the new Texas state budget proposal currently under review in the legislature
Grok does a good job here of telling exactly where we are in the process: the House has approved one budget; the Senate has approved another; they are then going to have to hash out the differences in a conference committee.
Notice, however, that Grok’s answer focuses on the differences between the House and Senate versions. It doesn't address how these proposals compare to the existing budget, which is what we really need to give reader a sense of how this budget will affect their lives. That’s our fault; our prompt didn’t specifically ask for this, so Grok set up its own comparisons. So let's write a more structured, detailed prompt and see if we can get better results:
Write a three paragraph summary of the new Texas state budget proposal currently under review in the Legislature compared to the current Texas budget passed in 2023
We do get better comparisons, such as noting that the proposed budget allocates $134 billion for education, up from $128 billion in 2023 budget; the $16 billion boost for public schools; and $51 billion in property tax relief. This response details key differences in specific priorities and funding mechanisms. It’s a much better result than what we got from Grok with the first prompt, and much more useful as the background in a budget story.
Now let's see what sort of results we get if we treat Grok like a cub reporter and give extremely detailed instructions as part of the prompt:
Write a summary of a comparison of the Texas state budget proposal found here:
https://www.lbb.texas.gov/Documents/Appropriations_Bills/89/House/HseFull_Cmte_GAB_89R.pdf
With the existing budget found here: https://www.lbb.texas.gov/Documents/GAA/General_Appropriations_Act_2024_2025.pdf
Provide three bullet points with the largest changes in dollars from 2023 to 2025, and three bullet points with the largest percentage changes from 2023 to 2025.
Note that we're asking for a lot more specificity here. More importantly, in the first two prompts we let Grok decide what was important. Now we're giving it specific constraints: what are the three largest changes in dollars? What are the three largest percentage changes?
The writing is mechanical at best, but that’s a pretty good, solid background section for a budget story. Indeed, I’d rewrite this to reverse the order: start off the section with one of the bullet sets, then slide into the overall background. But that’s the work of minutes, rather than the hours and hours of work it would take to read through the hundreds of pages of a budget, feed it into a spreadsheet, and run comparisons.
And obviously we can fine-tune all this by adding detail and structure to these prompts ad nauseam, and getting better and better results.
Even a fully staffed newsroom won’t be able to compete with the efficiency possible with Virtual Newsroom for stories like this. Grok responded to each prompt within seconds.
The current budget proposal is — I'm not making this up — 1,023 pages.
Of numbers.
Just reading that takes hours. And the old budget is of a similar size. Even with an existing spreadsheet built for the last budget cycle, going through 1,023 pages, cranking all that into the spreadsheet and comparing it to the old budget takes hours.
If Virtual Newsroom can run those comparisons and crank out background paragraphs in seconds, that frees reporters up to go out and do actual reporting, to focus on interviewing and asking the hard questions, rather than just the mechanical part of comparing two thousand-plus page budgets to each other.
Virtual Newsroom can quickly and efficiently summarize what happened. That frees up the journalists to go and ask why it happened. Why is the increase in Property Tax Relief bigger than the increase in Education? Why is Property Tax Relief a higher budgetary priority than Education?
I think that's the strength of a thing like Virtual Newsroom.
After working on Virtual Newsroom for a year, I now think it is not a software solution. Instead, I think what we need to do is recreate the type of efforts that marked the early days of the Computer-Assisted Reporting and Research movement. We need to build a Virtual Newsroom toolbox, much the same way as computer-assisted reporting and research was a toolbox. There will be some basic software, but more importantly it will be collections of templates and training and templated prompts like this one we built for stories about the Texas biennial budget.
In order for small newsrooms to be viable with just one or a few journalists, those journalists need to be free to focus on journalism. It does no good to have tools like Large Language Models available, no matter how effective they are, if using them requires those same journalists to spend time learning prompt engineering and building templates.
A community trading templates and training and best practices could make every journalist more effective. I think that is what we're building toward here and will give the ability of the smallest newsrooms to do mighty, mighty work.