On Probabilism and Determinism in AI

by Narain Jashanmal

Introduction

An AI development framework characterized by shifts between deterministic and probabilistic approaches in computing. It outlines four stages:

We will analyze the historical accuracy of this framework, its logical coherence, and the future implications of reaching the final stage. We'll also identify companies and technologies working toward that future (Probabilistic → Probabilistic → Deterministic).

Evolution of AI Paradigms

Early Era: Fully Deterministic Systems (Deterministic → Deterministic → Deterministic)

In the earliest stage of modern computing, everything was deterministic. Both the input data, the programmed algorithm, and the output were fixed and predictable. In practical terms, this era includes tools like spreadsheets and relational databases. For example, a spreadsheet (like Microsoft Excel) will always produce the same result given the same set of inputs and formulas – if you input X and apply a set formula, you deterministically get Y every time. Classic databases (Oracle, IBM DB2, etc.) similarly execute queries with deterministic algorithms (e.g. SQL joins, filters) to return precise, unchanging results for a given query.

Traditionally, computers have been deterministic machines, providing consistent outputs for the same input. In other words, early software was engineered such that if you ran the same operation twice with identical inputs, you'd get identical outputs. This property was (and still is) essential for applications like financial calculations or transaction processing.

Historical accuracy: Describing the start of computing as deterministic is accurate. From the mid-20th century through the 1980s-1990s, most software followed explicit rules and logic. Even early "AI" in that era (expert systems) were rule-based – essentially a series of hardcoded deterministic if-then statements. Companies like Microsoft (with Excel) and Oracle (with database systems) dominated this deterministic computing era. These systems had no randomness involved: their behavior was fully determined by their code and input, reflecting the Deterministic → Deterministic → Deterministic model.

Big Data and ML Era: Probabilistic Algorithms with Deterministic Outputs (Deterministic → Probabilistic → Deterministic)

As data grew and problems became more complex in the 2000s-2010s, we entered an era where probabilistic algorithms (machine learning models) were introduced into otherwise deterministic systems. In this stage, the inputs were often well-defined (e.g. a user's profile or a search query – which we can consider "deterministic" in the sense that the input is a fixed piece of data), but the processing was done by probabilistic models. However, the system would still produce a single decided output – giving the appearance of determinism in the final result.

A prime example is online advertising and content recommendation. Companies like Google and Meta (Facebook) led this shift. Google's search and ad products traditionally took a user's query (deterministic input) and then, starting in the mid-2000s, used machine-learned models to rank results or choose the best advertisement. These models compute things like the probability of a user clicking each ad or finding a result relevant. The highest-scoring result is then shown to the user as the answer or ad – a single, discrete output. The user doesn't see probabilities; they just see one ad or one set of search results. In other words, internally the system is probabilistic, but externally it delivers a fixed outcome.

Historical accuracy: Characterizing the 2000s-2010s as adding a probabilistic middle to otherwise deterministic systems is largely accurate. Advertising technology is a clear example – over time it moved from deterministic rules (show ads based on fixed keywords or demographics) to probabilistic predictions (show ads based on likelihood of engagement). By the late 2010s, virtually all major internet platforms were using machine learning models under the hood.

Current Era: Generative AI and End-to-End Probabilistic Systems (Probabilistic → Probabilistic → Probabilistic)

Today, with the rise of Generative AI (large language models, diffusion models for images, etc.), we have probabilistic behavior at every stage. The input might be an open-ended prompt from a user – for example, "Write a story about a dragon." This input is not a structured, unambiguous piece of data; it's a probabilistic or wide-ranging input in the sense that it could be interpreted in many ways (the AI has to decide which direction to take the story, what style, etc. based on learned probabilities). The model (like OpenAI's GPT-4 or Anthropic's Claude) then probabilistically generates text by selecting likely next words according to its training. Finally, the output itself is probabilistic: it's not a single fixed answer but one of many possible valid outputs. If you ask the same question multiple times, a generative AI might give different phrasing or even entirely different answers, especially if any randomness ("temperature") is involved in generation.

Large language models can produce slightly different responses for the same prompt due to the use of randomness and probabilities during generation. If you prompt GPT-4 or ChatGPT twice with exactly the same request, you may get variations in the reply. This stands in contrast to traditional software which would give the exact same output every time. This "controlled fuzziness" mimics human-like variability and creativity.

However, having probabilistic behavior at all stages also introduces challenges. The output might be plausible yet incorrect – what we call AI "hallucinations" or errors. Stephen Wolfram illustrated this by asking ChatGPT to compute a factual question ("How far is it from Chicago to Tokyo?"). ChatGPT produced a convincing-sounding answer, but it turns out that it's wrong, whereas a deterministic computation via Wolfram|Alpha got the correct distance. ChatGPT's responses are always statistically plausible but not guaranteed true. The model will confidently "trot out" facts or figures that aren't necessarily correct because it's drawing on patterns, not a grounded database.

Historical accuracy: Describing the current state of AI as Probabilistic → Probabilistic → Probabilistic is valid. Around 2016–2023, deep learning models grew in ability, and by late 2022 the public saw ChatGPT's emergence, highlighting the paradigm of fully probabilistic AI. The framework's characterization of "where we are now" aligns with how generative AI operates. Major players (OpenAI, Anthropic, Google with Bard, etc.) are indeed using probabilistic neural networks to handle ambiguous inputs and produce open-ended outputs.

Future Outlook: Toward Probabilistic Inputs/Algorithms with Deterministic Outcomes

The proposed final stage is Probabilistic → Probabilistic → Deterministic. This implies an AI system that can accept the messy, probabilistic inputs from the real world, process them with advanced (likely probabilistic or learned) techniques – but still guarantee a reliable, deterministic output. In essence, it's the best of both worlds: we want AIs that can understand and reason about uncertain, complex situations (like humans can, using probability and intuition), yet deliver results that are as dependable and verifiable as traditional software.

Why do we "want to get to" this stage? One big reason is to combat the flaws of current generative AI, such as hallucinations and inconsistency. If an AI's final output can be made deterministic (correct and the same every time for the same query), we can trust it in high-stakes applications. Industry reports note that the risks of hallucinating fictional summaries will be better managed as algorithms evolve from probabilistic to deterministic outcomes.

Approaches Being Pursued

Several companies and research efforts are effectively aiming for Probabilistic→Probabilistic→Deterministic systems:

Who Is Working Toward This "Final Stage"?

Future implications: If we achieve Probabilistic → Probabilistic → Deterministic AI, it could revolutionize trust and adoption of AI in critical fields. Imagine an AI lawyer that can take a fuzzy description of a case, use a learned model to interpret and reason about it, but then output a deterministically correct legal brief that cites all the right laws and never fabricates a case. This could unlock using AI for autonomous driving, financial advice, scientific research, etc., where today we are cautious because of unpredictability.

Conclusion

The proposed framework of AI development – from fully deterministic beginnings, through a hybrid probabilistic phase, into today's fully probabilistic generative models, and onward to hopefully more deterministic outputs – holds up under analysis. Historically, it's broadly accurate. Logically, the progression makes sense: each stage addressed the limitations of the previous. The ultimate goal of Probabilistic → Probabilistic → Deterministic is essentially about keeping the intelligence and flexibility of modern AI while re-imposing the reliability of earlier software.

If successful, this convergence could yield AI systems that are both powerful and trustworthy – capable of human-like understanding and creativity, yet delivering answers and actions we can count on with the certainty we expect from a computer. Achieving that will likely define the next era of AI development, unlocking new applications and increasing societal trust in AI.