LLMs and Genuine Discontinuities: Hume’s Problem of Induction

In this essay, I examine how Hume’s problem of induction, his argument that prediction relies on habit rather than rational certainty, explains why Large Language Models (LLMs) reproduce historical regularities and therefore struggle with genuine discontinuities. Historical regularities are stable, recurring patterns from the past. Regular variations allow a pattern to continue, making inductive prediction practically useful even when particular outcomes differ. A discontinuity interrupts an existing pattern, while what I call a genuine discontinuity changes the assumptions underlying that pattern and may introduce a new framework. For example, uncertainty about who will become the next king is a regular variation, a challenger interrupting the expected succession is a discontinuity, and the replacement of the monarchy with a democratic system is a genuine discontinuity. I argue that Hume’s problem of induction exposes an epistemological limitation of LLMs: they rely on patterns inferred from historical data even though the continued reliability of those patterns cannot be rationally demonstrated when the underlying conditions change. Hume divides the objects of human reason into two categories: relations of ideas and matters of fact (Hume, 1748, Section 4). Relations of ideas are intuitively or demonstratively certain and can be established independently of experience. Their method is deductive: if the premises are true and the reasoning is valid, the conclusion must necessarily be true without requiring empirical observation. For example, a square has four sides and four right angles. We do not need to observe or experiment on every possible square to know that this definition remains true regardless of the square’s size. By contrast, matters of fact concerns what exists or occurs in the world and depend on experience. Their opposites are logically conceivable because denying them does not produce a contradiction. For example, yesterday I observed dark clouds followed by rain. When I see dark clouds today, I may inductively infer that it will rain again. However, it is still logically possible that it will not rain. The prediction may be reasonable based on past experience, but its truth is not deductively necessary.

Hume argues that reasoning about matters of fact beyond present observation and memory depends on the relation of cause and effect. This reasoning assumes that the future will resemble the past, an assumption commonly called the Principle of the Uniformity of Nature. However, this principle cannot be rationally justified through either branch of Hume’s Fork. First, it cannot be proven deductively because denying the uniformity of nature creates no logical contradiction. It remains conceivable that an established pattern or even a law of nature could change tomorrow. Second, the principle cannot be justified inductively. Arguing that nature will remain uniform because it has always appeared uniform in the past is circular because it uses the past reliability of induction to establish its future reliability. The argument therefore assumes the very principle it is attempting to prove. Hume concludes that our causal expectations are not grounded in deductive proof but arise through custom or habit. After repeatedly observing one event followed by another, a constant conjunction between them, the mind naturally comes to expect the second event when the first occurs again (Hume, 1739–1740, Book 1, Part 3, Section 6; Hume, 1748, Sections 4–5).

LLMs operate in a way that resembles Hume’s account of how humans reason about matters of fact when deductive certainty is unavailable. Rather than establishing their outputs through the logical necessity found in relations of ideas, LLMs learn statistical regularities from large collections of historical text during training. These regularities are represented in the model’s numerical parameters. During generation, an LLM uses relationships among tokens, which may represent whole words, parts of words, punctuation, or other character sequences, to estimate a probability distribution for the next token (Radford et al., 2019). For example, an LLM may encounter many sentences such as “The dog barked,” “The dog chased the cat,” and “The dog is sleeping.” From repeated examples, the model learns grammatical, semantic, and stylistic patterns associated with the word “dog.” This process does not guarantee that the generated statement is true. It identifies which continuation is probable based on patterns represented in the model’s training data and the context supplied by the user. In this respect, an LLM’s operation resembles inductive inference. So, it applies patterns derived from previous cases to a new and unobserved case. This process functions as though previously learned linguistic and factual patterns will remain applicable to new cases. Therefore, it resembles the Principle of the Uniformity of Nature associated with Hume’s problem of induction, although an LLM does not consciously accept this principle. In this analogy, the expected uniformity concerns patterns in language and recorded information rather than only causal regularities in nature. Applying past linguistic patterns is often reliable when language changes gradually, reflecting what I call regular variation. For example, the word “awful” once commonly meant “inspiring awe,” whereas it now ordinarily means “very bad.” An LLM trained on texts from both periods may distinguish between these meanings by using historical and linguistic context. This semantic change is not necessarily a genuine discontinuity because usages of both meanings may be represented in the training data. Nevertheless, the example illustrates that linguistic patterns are not permanently uniform. Meanings can vary across time and place, making a learned pattern reliable only under particular historical and linguistic conditions.

A more significant epistemological disruption occurs when an accepted conceptual framework is replaced. The historical transition from geocentrism to heliocentrism did not alter the physical structure of the universe. Instead, it transformed the framework through which people interpreted astronomical observations. A model trained primarily on texts written when geocentrism was dominant might reproduce geocentric claims because they were statistically prevalent, even though their prevalence did not establish their truth. New observations and arguments eventually challenged this framework and contributed to the acceptance of heliocentrism. This transition demonstrates that repetition and consensus in historical records cannot guarantee accurate knowledge. In this case, the genuine discontinuity was epistemological rather than physical: the framework used to explain existing observations changed. A genuine discontinuity therefore presents a difficulty not merely because an outcome is unusual, but because the assumptions used to interpret and predict outcomes have themselves changed.

An LLM encounters the epistemological limitation identified by Hume when it applies historical patterns to situations in which the underlying conditions have changed. This does not mean that an LLM must always be retrained before it can discuss a new event. New evidence may be supplied through a prompt, a retrieved document, or another external source, allowing the model to respond without changing its trained parameters. However, the model cannot identify an unobserved event as an established fact before relevant evidence becomes available. Retraining or new contextual information may help the model adapt after a discontinuity, but neither can prove that the new evidence is accurate or that the newly learned pattern will remain reliable in the future. The main problem is therefore not that LLM predictions are always false or useless. It is that success with past and present patterns cannot rationally guarantee reliability under genuinely new future conditions. At this point, we might question whether the problem of induction under genuine discontinuity is different for LLMs than it is for humans. Humans also struggle with genuine discontinuities, especially when inductive patterns have developed into long-term habits. Hume’s argument is therefore universal and applies to both humans and LLMs. However, there is an important practical distinction. Humans can directly observe the world, conduct experiments, manipulate physical objects, investigate exceptions, and test competing explanations. An unaided, text-based LLM cannot perform these activities independently. It adapts when new information is supplied through its training data, context, retrieved sources, or external tools, and then generates new statistical probabilities from that information. This supports the claim that LLMs are epistemically dependent on external sources when confronting genuine discontinuities. If an LLM is connected to sensors, scientific instruments, or other AI systems, this distinction becomes narrower. Nevertheless, neither humans nor artificial systems escape Hume’s problem, because even new observations cannot deductively guarantee that the patterns they reveal will continue.

Another objection is that a pattern may remain practically useful even if its future reliability cannot be proven. Like humans, LLMs do not require certainty to make predictions. Instead, a sufficiently high probability may be enough for practical purposes. After all, science relies extensively on induction and has made significant progress. However, scientific hypotheses are also subjected to observation, experimentation, criticism, and revision. An unaided, text-based LLM cannot independently perform these activities to verify that a new explanatory framework corresponds to reality. It depends on observations, experimental results, or testimony entering its training data or current context. For example, imagine an LLM trained on many descriptions of objects falling but given no previous explanation of gravity. The model might identify the regularity and even propose gravity as a possible explanation, but it could not independently conduct experiments to determine whether that explanation is better than its alternatives. Furthermore, it would still assume that the newly identified gravitational pattern will continue in the future. LLMs may therefore remain useful for predicting regular variations and generating hypotheses while still possessing the epistemological limitation identified by Hume, especially when confronting genuine discontinuities.

In conclusion, LLMs are subject to Hume’s problem of induction because they rely on patterns inferred from historical data. Genuine discontinuities expose the limitation of inductive reasoning because past regularities may no longer apply when the underlying framework changes. However, induction remains practically useful in the large number of cases where conditions remain sufficiently stable, even with regular variations or limited discontinuities. Furthermore, LLMs, even when text-based, can use inference to develop hypotheses and propose possible solutions to novel problems, similar to what humans do in science. However, proposing a hypothesis does not empirically validate it. When LLMs have access to external sources containing new observations, experimental results, and other evidence, they can update their responses as the world changes. Users must therefore recognize the difference between a statistically plausible prediction and an empirically tested conclusion. A probable conclusion is not a deductively demonstrated truth, but both humans and LLMs can still use induction as a practical tool while recognizing its epistemological limits.

APA References

• Hume, D. (1739–1740). A treatise of human nature. Hume Texts Online. https://davidhume.org/texts/t/
• Hume, D. (1748). An enquiry concerning human understanding. Hume Texts Online. https://davidhume.org/texts/e/
• Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI. https://cdn.openai.com/better-language-models/language_models_are_unsupervised_multitask_learners.pdf