Does a Recommender System Know Me Better Than I Know Myself?

How a Person Builds a World · Season Three: Attention, Desire, and Self-Understanding · Article 10

An unfamiliar song begins and happens to fit the present mood. An article appears in a list on the very question one has recently considered without ever searching. We respond with surprise: “It knows me better than I know myself.” Half the sentence praises predictive accuracy; the other half merges two different meanings of knowing.

A recommender system can identify patterns in large records of behaviour that a person cannot notice unaided. It may predict the next click, view, purchase or interval of attention with impressive accuracy. Predicting an act, however, is not the same as understanding why a person cares, what they hope to become, or which short-term impulses they do not want to continue. More importantly, what a system presents becomes an input into future behaviour. It does not merely observe a completed preference. It participates in the formation of preference through a feedback loop.

The system sees behavioural traces, not complete desire

Recommendation ordinarily requires turning observable activity into signals: clicks, watch time, skips, searches, ratings, purchases, or what similar users also consumed. Platforms, models and objectives differ substantially, so “the algorithm” should not be treated as one agent. A common limitation remains: a system can learn only from traces that are recorded and defined as useful.

Dwell time may indicate appreciation, or it may indicate confusion, anger or difficulty leaving a page. Completion may mean background playback. Absence of a click may mean that the item was never displayed rather than that the user lacked interest. A recommendation model need not settle these questions of meaning. It can operate effectively whenever a signal improves prediction of the selected objective.

“User preference” within the model is therefore first an operational representation built for a task. It may be highly useful without amounting to a person’s complete statement of value. If someone repeatedly watches undemanding clips late at night, the system has reason to predict another. That does not answer whether the person wishes to allocate more future evenings to the same activity.

Once an option has been recommended, the choice is no longer pure evidence of prior preference

Recommender systems learn from past behaviour, but past behaviour has already been affected by past recommendations. Researchers call the resulting problem a feedback loop or algorithmic confounding. A user accepts a recommendation; the behaviour is recorded; the record trains the system; and the system becomes more confident in presenting similar items.

In a 2018 simulation study, Allison Chaney, Brandon Stewart and Barbara Engelhardt analysed algorithmic confounding and found that repeated recommendation and feedback could increase homogeneity and, under their model conditions, reduce utility for some users. Other researchers have attempted to separate the influence of recommendation from so-called intrinsic preference in observed ratings. These results come from particular models and datasets. They do not prove that every real platform inevitably traps every user in a filter bubble. They expose an identification problem: behaviour later used to prove “you like this” may exist partly because the system previously ensured that this was what you saw.

Feedback is not necessarily harmful. A friend’s recommendation changes taste. A teacher’s selection of materials develops capacities. Repeated exposure can make unfamiliar art intelligible. Preferences have always developed within environments. The issue is whether the direction optimised by the system corresponds to the person one hopes to become, and whether the user can see and alter that direction.

Accuracy depends on which moment of the self is being predicted

A recommendation that obtains a click within five seconds can be called accurate. A playlist that prevents departure for an hour can be called successful. The person’s own aim may instead be to encounter unfamiliar work, finish a task, improve sleep or stop being repeatedly captured by content that induces anger. Accuracy for immediate reaction is not identity with long-term value.

Recommendation also confronts multiple selves. Searches performed for work, a child’s viewing on a shared device, one-off travel planning and long-term interests enter the same record. A click of curiosity and an endorsement of a position may look identical. Models can improve distinctions through time, devices and context, but behaviour alone cannot decide which tendency more truly represents the person. The question itself contains a normative judgement.

When we say a system knows us better, it may indeed predict a micro-behaviour better than we do. People are not transparent to themselves and often underestimate habit. The system’s advantage, however, arises from narrowing its task. It does not have to explain a life. It needs to rank the next item within a bounded set of candidates. Converting an advantage in local prediction into an understanding of personhood is an excessive philosophical translation of technical performance.

A recommendation interface should let people express preferences they do not want to continue

Many controls permit only “like”, “dislike” or “not interested”. Self-understanding requires a richer language of time: I need this today but do not want it to define long-term recommendation; I am researching this, not endorsing it; I want less of this even though I have clicked it often; I want unfamiliarity rather than greater similarity.

Good control over recommendation should do more than permit deletion of history. It should help users understand which activities influenced the system, distinguish temporary from enduring aims, set diversity and stopping conditions, and allow a preference signal to decay. Explanation need not expose an entire model’s code. What matters personally is actionable information: “because you watched several items on this subject recently”; “reduce this source”; “do not use activity during this period to train your recommendations”.

A system also has a responsibility to explore. If only the highest predicted item is repeatedly placed first, an untried interest will never generate data, and lack of exposure can be mistaken for lack of preference. Preserving some diversity and chance is not merely a way to make entertainment more pleasant. It prevents a person’s past from acquiring a monopoly over the future.

The user, in turn, can treat a recommendation as a mirror rather than a portrait. It shows which patterns their behaviour has left and which are strong enough to predict a next move. An unexpected recommendation can prompt a question: has the system found an interest I had not named, or captured a vulnerable moment I do not want to extend? Is the recurring content evidence of stable choice, or path dependence jointly produced by the platform and me?

The system knows how to continue; the person must judge whether to continue

The knowledge in which recommendation excels is continuity: given the interactions that have occurred, what is most likely to sustain another interaction? Self-understanding must also contain the capacity to interrupt continuity. A person can acknowledge, “I really will click it”, while saying, “I do not want this predictability to organise my future.” That refusal is not an error in the data. It is a commitment.

A recommender may consequently know my behavioural pattern better than I do within a narrow task. It does not follow that it understands me better. Fuller understanding includes reasons, conflict, time, consequences, other people and revisable value. None can be inferred automatically from the next click.

The more accurate relation is neither that the individual owns a finished authentic preference which the system merely discovers, nor that the system manufactures everything while the user has no agency. Preference is expressed, reinforced, revised and created in a loop between person and recommendation environment. Precisely because a system participates in formation, it should not present its output as a neutral reading of a fixed self.

A trustworthy recommender must do more than predict what I will continue. It should let me state which parts of the past should not continue. A person capable of using recommendation without being closed by it must ask more than whether the guess is accurate: “What kind of world is this guess helping me to form?”

Primary sources and further reading

• Allison J. B. Chaney, Brandon M. Stewart and Barbara E. Engelhardt, “How Algorithmic Confounding in Recommendation Systems Increases Homogeneity and Decreases Utility”, RecSys, 2018.
• Ayan Sinha, David F. Gleich and Karthik Ramani, “Deconvolving Feedback Loops in Recommender Systems”, NeurIPS, 2016.
• Marko Tkalčič, Markus Kunaver et al., “Towards Psychology-Aware Preference Construction in Recommender Systems”, Journal of Intelligent Information Systems, 2022.
• Paul Slovic, “The Construction of Preference”, 1995.

Continue reading: Explore the How a Person Builds a World series.