Are there any particular areas of research you’ve found challenging?

AC: We’re exploring areas like healthcare and infrastructure, and more durable goods like the value of having a car. What we’re finding in our survey-based approach is that it’s more challenging to measure goods that are essential to survival.

It’s easier to put values on goods when you can live without them. But if it’s impossible to give up something, the perceived values are, of course, very high. People have difficulty saying if it’s 1 million dollars or 10 million dollars, so when the numbers become too high our method doesn’t work as well.

One approach is to focus on changes over time rather than the absolute numbers. How much one values healthcare is a hard question because it’s essential, but the value of the improvement in healthcare from last year to this year is relatively easier.

DN: Since I first started working on economic measurement under Professor Diane Coyle, I realized quickly how large the challenges lying ahead of us are. Now I know that it is possible to come up and establish new indicators if you are willing to build the networks. Walking the line between improving traditional metrics that remain relevant and establishing something complementary can, at times, be challenging. 

What are the areas of concern when looking to apply GDP-B internationally?

DN: We are working with other countries to establish GDP-B as a global indicator from the start. There are no theoretical barriers to using GDP-B as an internationally comparable indicator. 

AC: When we measure GDP contributions from digital goods across several countries, we find lower-income countries benefit much more from digital goods than higher-income countries. Similarly, within a country, lower-income individuals seem to benefit much more from digital goods related to their income compared to higher-income individuals. 

It’s not like Elon Musk gets a better quality Google search, right? Elon gets the same Google search as you and I, and for free provided you have internet access. If you are richer, you have better homes, better cars. But in a digital world, if you are rich or poor, you still have the same Google search, the same YouTube, and so on.

One caveat I want to mention is this is not to say digital goods are amazing. We are only looking at the economic benefits. If you measure happiness and subjective well-being, there could be negative impacts as well.

Does GDP-B examine any potential harms or negative impacts?

AC: It’s hard to measure these externalities. For example, we find that social media apps generate a lot of welfare and billions of dollars, but–this is a question of debate in academic circles–they may impact increasing political polarization at the national level. That wouldn’t show up in our data because we are measuring individual-level benefits. 

DN: We are actively working on accounting for the downsides of (over-)consumption like addiction, and defensive expenditures like cleaning up after environmental disasters or conflicts. GDP has the same issues given that many things we might see as welfare decreasing are counted as positive factors. Examples include cigarettes, surgeries after preventable accidents, or securing private property.

SK: From a purely economic standpoint, standard GDP rarely factors in costs like pollution or other negative externalities; it just measures output or consumption. In the same vein, GDP-B zeroes in on consumer benefits. So, GDP-B only captures the upside.

However, this is an important point that we keep coming back to. We would need to develop a methodology to capture potential harms, perhaps something along the lines of surveying users on how much they’d pay to avoid data privacy issues or unwanted attention “taxes.” But these types of questions are more abstract and are inherently more difficult to answer. Ask yourself, how much would you be willing to pay to not feel the urge to check your phone every five seconds?

How do you see policymakers using these new measures?

SK: I think policymakers could use these new measures (like GDP-B) to inform decisions about everything from regulating platform services to prioritizing infrastructure that boosts overall welfare, not just the economy’s “cash flows.” They’d have numbers that more accurately reflect the actual quality of life and benefits that people experience, especially in the realm of “free” digital goods.

As for the general public, a measure that captures the free services they actually use might resonate better than GDP alone, which can feel disconnected from everyday experiences. People might find it validating to see their reliance on messaging apps or streaming services recognized as part of the economy’s real value.

AC: Looking at changes in GDP-B over time would be a great proxy to see, are people better off or worse off because of various technologies?

Let’s say a policymaker is thinking about regulating generative AI technologies. We can, using GDP-B numbers, put a dollar number on what would happen, an increase or decrease in benefits, from a potential regulation. These numbers would help them come up with better policies because they directly capture the benefit.

If you could ensure one specific change in how we measure economic progress in the next decade, what would it be?

AC: We would have a “dashboard” of metrics–this is something Erik Brynjolfsson said a few years back. If you’re driving a car, you have a dashboard, and you have different numbers. We have speed, we have fuel, all these different numbers. 

Similarly, for policymaking, we need a dashboard. We already have GDP and productivity. In addition to that, we can measure happiness. We create a dashboard, and we try to focus on looking at this spectrum rather than focusing on one specific number. This would be on my wish list for the future.

DN: GDP-B becomes the default global measure of progress and societies are benchmarked against improving the things that people ultimately care about. 

SK: Personally, I’d push for routinely measuring consumer surplus from digital goods—how much value people actually get from services that don’t have a price tag, like Google Maps or Wikipedia. This is something that’s long overdue.

Have you seen any sort of pushback?

AC: Whenever we present GDP-B, it has been very well received. The big challenge, of course, is that statistical agencies–in the US the Bureau of Economic Analysis–have been facing declining budgets over the past several years. They are already struggling to produce the existing GDP numbers. This is a new method requiring survey data collection, which costs a lot of money.

DN: Policymakers have shown a keen interest in our work, demonstrating that there is a demand out there for welfare indicators beyond GDP. One challenge is the decreasing budget and funding for economic measurement and statistics at the federal level. This is ill-guided given that the rapidly changing economic structure requires better measurement techniques to keep track of what is going on.

What steps are being taken to future-proof GDP-B?

SK: The focus needs to be less on specific tools, and more on the framework. What people value changes fast these days, and our focus needs to be on that. 

DN: GDP-B is a flexible measure that can easily account for new products in a rapidly changing digital economy – much more so than traditional GDP that struggles whenever the product mix changes.

AC: We’re creating a template for running the service. If someone is interested in digging deeper into a specific industry or geography, they can take our template, do their own surveys, plug the numbers into our framework and come up with GDP-B estimates for their country or region.

The real impact will come from adoption. If countries adopt GDP-B, that’s where the scale will come.

You can learn more about GDP-B by visiting gdp-b.org.

***

Avinash Collis is an Assistant Professor at the Heinz College of Information Systems and Public Policy at Carnegie Mellon University. Avi holds a Ph.D. from the MIT Sloan School of Management. He studies the economic implications of information technologies.

David Nguyen‘s research explores new and better ways to measure the modern and digital economy. He is particularly interested in advancing economic metrics and statistics on economic output and welfare. As a research associate, he remains affiliated to the London-based Economic Statistics Centre of Excellence (ESCoE). David received his PhD from the London School of Economics.

Sophia Kazinnik is a research scientist at the Stanford Digital Economy Lab, where she explores the intersection of artificial intelligence and economics. Prior to joining Stanford, Sophia worked as an economist and quantitative analyst at the Federal Reserve Bank of Richmond, where she was part of the Quantitative Supervision and Research group. While there, she contributed to supervisory projects targeting cyber and operational risks and developed NLP tools for supervisory purposes

Better AI Can Reduce the Risks

If poorly designed AI can create risks, better AI offers a powerful way to reduce the risks of democratized statistics. 

Ideally, any AI helping with statistics will create a guided workflow.  First, it will automate data validation. Tools can flag issues such as missing values, duplicates, and outliers before analysis begins. Imagine an assistant that read in some data and reported: 

Your second variable is called height and appears to be in centimeters.  It also uses “9999” to code for a missing value.  One observation is 999 cm (33 feet) tall. It is likely that this value is a typo. Should I code it as missing? 

As the AI proceeds with the analysis, it can verify if the data satisfy statistical assumptions such as normality or linearity.  For example, before running a correlation or t-test, the AI could automatically check for normality and suggest a non-parametric test if necessary. 

It can also check functional forms. For example, it might ask:  

You requested linear regression, which assumes your variables have a linear relationship. But the residuals in your specification follow a predictable pattern (according to the Ramsey RESET test).  This result is often due to a nonlinear relationship.  I suggest using a more flexible specification such as polynomial regression and also an approach that does not make assumptions about functional form such as random forests. Should I run those analyses instead?

The AI system might also proactively alert users to potential methodological pitfalls. For instance, it could display a cautionary message such as, “Your dataset’s sample size is too small to derive reliable conclusions”—a feature already seen in some advanced software. Additionally, the AI could autonomously adjust P values in contexts involving multiple hypothesis tests, flag the risks of overfitting, or adjust standard errors to more accurately reflect the data characteristics.

One might also imagine the AI engaging interactively with the analyst to solicit further context or supplementary documentation. For example, it could request additional materials—such as codebooks, questionnaires, or survey instruments—to deepen its understanding of the dataset. If the analysis suggests that the available data are inadequate, the system might even recommend collecting additional observations to strengthen the study’s empirical foundation.

Perhaps most importantly, the AI assistant can help interpret the results.  To start, it can point out: “Remember, correlation does not imply causation. Consider additional analysis to understand causality.” 

As the AI system evolves, it will increasingly discern both the intent of the analyst and the contextual nuances of the problem. Such a well-informed assistant can go further in spotting and solving potential concerns.  For example, if the AI knows the analyst is examining the relationship of education and wages, it might add:  

It is important to consider a third factor, currently omitted from the analysis, such as the family’s emphasis on child-rearing that might increase both education and wages.

Similarly, it could warn about selection bias, such as: 

Because your sample consists entirely of individuals who have completed college or higher, these estimates may be subject to selection bias, as those who did not pursue post-secondary education are excluded. To properly interpret your results, it is crucial to account for this non-random selection process.

As the software learns from the user’s evolving expertise—and ideally, as the user’s own analytical skills expand—the degree of handholding and alerting provided by the AI can modify over time. AI systems can contribute to this process by logging each decision they make—from handling missing observations to selecting specific tests. Such an iterative approach can facilitate error detection, enhance understanding of the rationale behind the AI’s suggestions, and foster trust in the system. Just as researchers now routinely submit their Stata or R code with publications, one might foresee a future in which the submission of transcripts detailing LLM prompt-response interactions becomes a requisite component of the peer-review process.

How to implement? 

Many of these capabilities can be integrated simply by refining the prompt provided to a large language model.  For example, ChatGPT 4o implements some of these features if we add to a prompt requesting a correlation: 

To the extent possible, test whether the data satisfy the statistical assumptions of all techniques. Flag potential issues and interpret results with caution. 

Alternatively, it would also be possible to have additional software, module or agent that does this additional testing and explanation. In the context of increasingly popular agentic systems such functionality could be implemented as an adversarial mechanism: where one model is tasked with justifying each decision to another. However, relying on an extra module introduces a significant challenge: novice users, who stand to benefit the most from such extra guidance, may not recognize its relevance.

Thus, the optimal solution would be to embed this supplementary analysis as a default feature in any artificial intelligence tool that facilitates statistical analyses. Developers at organizations such as OpenAI, Anthropic, Google, and Meta should consider incorporating these instructive defaults into the system’s core functionality whenever it identifies that a statistical analysis is being performed. An effective approach might include querying users about their statistical proficiency at the outset and dynamically adapting the system’s feedback based on subsequent interactions and inquiries.

The Balanced Future of Statistics

As statistics become more accessible, the opportunities for innovation and insight are vast. However, the risks of misuse are just as significant. The same AI assistants that empower novices to use statistical techniques should also act as a safety net, helping new users avoid common pitfalls and guiding them to make confident, data-driven decisions.

By combining automated tools with ongoing education and awareness of statistical principles, we can create a world where everyone—not just experts—can harness the power of data responsibly and effectively. We will see users with less initial education producing rapid and credible analyses.  Along the way, the users will learn more statistics and all of society will benefit from more decisions informed by data.

As AI-based statistical tools mature, there is also an opportunity to integrate interactive learning modules. This integrated approach would allow users—regardless of their prior expertise—to progressively build deeper statistical literacy, thereby ensuring that the benefits of democratized data analysis are both far-reaching and sustainable.

We invite developers, educators, and policymakers to join the conversation and ensure that democratized statistics truly benefits everyone.

Note: Authors are listed in alphabetical order. The examples of age over 700 years and height of 999 cm are both from Levine’s experience. Written with the assistance of ChatGPT.

José Ramón Enríquez is a postdoctoral fellow at the Stanford Digital Economy Lab (Stanford HAI) and the Golub Capital Social Impact Lab (Stanford GSB). José Ramón obtained his Ph.D. in Political Economy and Government (PEG) from Harvard University in May 2023.

José Ramón studies the political economy of economic and political development with a focus on political accountability. Specifically, he has worked on understanding the role of information in improving political accountability, with a specific emphasis on misinformation, political polarization, and corruption; the causes and effects of criminal-political violence on democratic representation; and the effects of the lack of coordination across levels of government.

David I. Levine is a professor of business administration at Berkeley Haas and serves as chair of the Economic Analysis & Policy Group. Levine’s research focuses on understanding and overcoming barriers to improving health in poor nations. This research has examined both how to increase demand for health-promoting goods such as safer cookstoves and water filters, and how to change health-related behaviors such as handwashing with soap. He has also written extensively on organizational learning (and failures to learn).

David Nguyen is a research scientist at the Stanford Digital Economy Lab. David’s research explores new and better ways to measure the modern and digital economy. He is particularly interested in advancing economic metrics and statistics on economic output and welfare.

Prior to joining the Stanford Digital Economy Lab, David worked as an economist at the OECD in Paris, and as a senior economist at the National Institute of Economic and Social Research (NIESR). As a research associate, he remains affiliated to the London-based Economic Statistics Centre of Excellence (ESCoE). David received his PhD from the London School of Economics.

The modern-day Federalist Papers

In an effort to guide the critical decisions before us as a society, we borrowed the concept of The Federalist Papers, a series of essays written in the late 18th century that analyzed the great challenges of the day and provided a roadmap for institutional innovation for a young America. We concluded that it was time for another gathering of leading thinkers who could offer ideas about how we should approach governing in the age of AI. We asked 19 authors their thoughts and brought together their responses in a volume of essays we call The Digitalist Papers.

Unlike The Federalist Papers, we did not have a goal of persuading readers to adopt a pre-determined action (in their case, arguing for the ratification of the Constitution). Rather, our aim was to elevate the voices of varied experts who have informed and well-researched ideas and bring out a menu of solutions across a range of related topics. Authors represent diverse fields—including economics, law, technology, management, and political science—along with leaders from industry and civil society. 

This matters because the thoughtful examination of AI-enabled developments in relation to our institutions requires us to turn the spotlight on those institutions and ask whether they remain fit for purpose in all respects, and or if they can benefit from AI or other changes. This is a daunting exploration that demands deep domain expertise and collaboration across academic disciplines.

(One final thing worth noting: Authors did not see each other’s work before publishing.)

Here’s what we learned:

• AI has the power to make governance more inclusive and technologically integrated: A group of essays urged for a significant shift in how AI intersects with the democratic process. Ideas included AI-driven collaborative governance, using AI to scale direct democracy by amplifying citizens’ voices, and using digital tools to advance hyper-local community-focused engagement.
  
  
• AI is forcing us to rethink how we organize our government: Some authors focused on how our government’s failure to deliver the services its citizens have come to expect will lead to civic disengagement, and how the adoption of AI—as well as broader adoption of technology focused on meeting citizens’ needs—can help reinstill faith that government is for the people.
  
  
• The regulation of AI is arguably more important than the tech itself: Three authors focused on the role of regulation, stating that the processes by which we currently regulate don’t stand up to something as dynamic and transformative as AI, that we might be exaggerating on AI’s impact on information and thereby undermining trust in all media, regardless of its accuracy.

Given the broad spectrum of perspectives, at times the authors offered conflicting ideas and differing approaches. That is by design. But the bedrock of every essay was that democracy and AI can and should work well together; that our democratic institutions can be renewed and reinvented for the AI era.  

A technology as fast-changing as AI requires nimbleness, keeping an open mind, and the continual engagement of a diverse group of leaders to debate and guide the technology. It’s not inevitable that AI will lead to more freedom and participation in democracy. If left unchecked, there’s a chance AI would change how we govern in ways the Founding Fathers might have found abhorrent. Therefore, it is our responsibility to approach these urgent questions with the conviction that we have the agency to change it.

What excites you most about the potential for AI to enhance education?

TM: I believe this is the decade when AI can change education for the better. Why? 

First, we have known for a while that human tutoring significantly improves student learning. 

Second, we now have multiple online education platforms that have taught millions of online students, who therefore have more experience teaching than a person could get even if they taught for a hundred years. We now have machine learning methods that use that data to learn and teach better. 

Third, we have just begun to create new computer tutors that use LLMs like ChatGPT to teach in new ways, like having conversations with students about their confusions, and like co-writing essays with students. The opportunity is great, but we need to organize and fund this research if we want to turn this potential into reality.

EB: As an educator myself, I’m acutely aware that my industry has been badly lagging when it comes to using technology–after all, I often use a chalkboard that Aristotle himself would have been comfortable with. But with generative AI, we have the potential to transform education in a fundamental way. In particular, we know from research that students who get individualized tutoring can learn up to two standard deviations faster than students in a classroom where 20 or 30 kids are all instructed at the same pace. Of course, individual tutors have been far too expensive to provide for all kids. 

Now LLMs like ChatGPT have the potential to change that. I already know a lot of kids, and for that matter, adults, including a Nobel Laureate, who use these tools daily for individualized tutoring to learn new subjects and to dive deeper into existing topics. What’s more, LLMs can be incredibly engaging and entertaining. In the coming years, I expect a revolution as developers create even better versions of these tools and millions of stu dents gain their benefits.

Is there anything in particular you’d like readers to focus on?

TM: One of the most important things we can do for the workforce is give them a clear, real-time picture of how the demand for different worker expertise is shifting over time, and what education opportunities are available to them to chart their own career path. A key part of this is actually collecting this real-time information. That will require public-private data partnerships that combine data from different companies that have real-time information about job openings, resumes, salary scales, and more. 

We can do this, but it will require imaginative thinking about how to build these partnerships while respecting the legitimate privacy and competitive concerns of the organizations that have these data.

What would you hope to see if you did another report down the line?

EB: One thing I expect will happen if we do a report like this in the future is that we’re going to not just talk to talk but walk the walk. AI will be a much bigger contributor: helping us with the literature review, gathering the relevant data, analyzing it, and even assisting with the writing. We are acutely aware of some of the weaknesses and risks of AI, but also the real strengths, especially when it’s combined with human oversight.

Lastly, what do you hope people take away from the report?

TM: We strongly expect continuing technical progress in AI, and expect it to significantly impact the economy and the workforce. I hope readers of the report get that the future we’ll live in is not destined by technology—really, it will be determined by the decisions we make about how to use it. That includes decisions made by politicians, technologists, workers, and all of us.

EB: As we worked on this report, it became clear that there are some real opportunities for an amazingly better world when it comes to improvements in science, healthcare, and widely shared prosperity, but also a real risk in areas like privacy, bias, democracy, national defense–even catastrophic risk. We laid out different scenarios and discussed some of the choices that we can make individually and collectively. 

Our most important conclusion was that there is no predetermined future, but rather our choices will determine our future. In fact as the tools become more powerful, our choices become ever more consequential.