Reconstructing the Mathematical Paradigm of Knowledge-Work Performance Evaluation: Theory and Empirical Validation through a Stochastic-Process and Conditional-Expectation Framework
Hunan Vocational College of Commerce
Abstract
This paper argues that the stochastic process offers the most precise mathematical translation of the dynamism and uncertainty inherent in knowledge work, and it then subjects the resulting paradigm to empirical test. In the first part, we define performance as a stochastic process evolving within a complete probability space and invoke the measure-theoretic concept of the σ-algebra to delineate the information boundary of the organization. Drawing on contract theory, the dynamic-capabilities view, and the classical econometrics literature, we unify a wide array of seemingly disparate performance-evaluation methods into a single scientific optimization problem with a well-defined objective function: finding the estimator that minimizes mean squared error under a given information set—the conditional expectation E[X | 𝒢]. In the second part, a carefully designed "estimator-efficacy comparison experiment," conducted in an idealized "natural laboratory," systematically compares a novel computational estimator (DEPICT) against a ladder of benchmark estimators in their ability to approximate the optimal estimate E[X | 𝒢]. The empirical results show that an estimator's predictive accuracy is strongly associated with how well its capacity matches the complexity of the information set, and that incorporating a theory-driven, structured organizational knowledge base (OKB) markedly improves evaluation efficacy. Together, the two parts provide robust quantitative evidence for the validity of a new dynamic-capability-based performance-evaluation (DCPE) paradigm, and quantify the systematic bias that traditional evaluation tools incur through information loss.
Keywordsperformance evaluation · knowledge work · stochastic process · conditional expectation · mean squared error · management accounting · information set · computational management accounting
Introduction
From the control perspective of management accounting, the performance of knowledge work is not a static, deterministic number but a continuously evolving process that is inherently dynamic and uncertain.
First, an employee’s performance trajectory is a dynamically evolving curve. This dynamism is driven by the intrinsic evolution of individual capability. Arrow’s (1962) “learning by doing” model holds that individuals continually refine their knowledge structures through the accumulation of experience, progressively improving both the efficiency and the quality of their problem-solving; this learning process is typically nonlinear, marked by plateaus and bursts. In parallel, the dynamic-capabilities view of Teece et al. (1997) maintains that, in adapting to environmental change, individuals continuously sense, seize, and reconfigure, keeping their capability portfolios and performance patterns in flux. Moreover, in rapidly changing technological environments, knowledge is subject to pronounced depreciation and forgetting. Traditional evaluation mechanisms compress a continuous process into a single point in time, introducing severe time aggregation bias and leaving the system unable to distinguish short-term sprints from long-term capability growth (Ittner and Larcker, 2003).
Second, the value-creation process of knowledge work is replete with unpredictability. This uncertainty can be separated into two kinds. The first is aleatory uncertainty, which arises from the inherent randomness of the system. In his work on agency theory, Holmström (1979) emphasized the signal-to-noise problem of performance signals: the randomness generated by innovation, together with disturbances from the external environment, injects uncontrollable noise into the system. The second is epistemic uncertainty, which stems from the tacit character of knowledge and from causal ambiguity, so that an organization’s knowledge of an individual’s true capability state remains perpetually limited. Building on the “controllability principle” of management accounting, Demski (1976) argued that an individual should be held responsible only for performance within their span of control. Yet in highly uncertain environments it is exceedingly difficult to separate controllable factors (signal) from uncontrollable ones (noise) with precision. Ignoring this uncertainty causes random “good luck” to be misread as “high ability,” producing distorted incentives and organizational injustice.
Variable Formalization
To characterize this complexity with minimal loss of information, performance must be defined rigorously within a probabilistic framework. Assume a complete probability space (Ω, ℱ, P):
- Ω (the sample space) represents the set of all possible future scenarios and historical paths.
- ℱ (the event space) represents the set of all theoretically measurable events, constituting the cognitive structure imposed upon uncertainty.
- P (the probability measure) is the function that quantifies uncertainty, assigning a probability of occurrence to each measurable event.
On this space, employee performance is modeled as a stochastic process {Xt, t ∈ T}, where the index set T may be discrete or continuous. For any instant t, Xt is not a single number but a function mapping a particular scenario to a concrete performance value; observed real-world performance is merely one realization of this process. This shift moves the object of evaluation away from a single-point “true value” and toward the dynamic distributional features of the process—for example the mean function E[Xt] (reflecting the growth trend), the variance function Var[Xt] (reflecting risk or stability), and the autocovariance function Cov[Xs, Xt] (reflecting path dependence).
Combining the knowledge-based view (KBV) with the dynamic-capabilities view, the performance-generating mechanism underlying this stochastic process can be formalized as a standard state-space model.
Observation equation
State-transition equation
Here the capability-state vector Kt (a latent variable) evolves as a function of its prior state Kt−1, current learning investment Lt (such as training and practice), and a random term ηt. The observation equation describes the mechanism by which Kt, moderated by a task-context vector Ct, is transformed into observable performance Xt. Owing to the complexity of knowledge integration, the performance-generating function f(·) necessarily contains an interaction term between capability and context (Kt × Ct) and is typically nonlinear, nonconvex, and marked by substantial individual heterogeneity. Together, these features cause the performance process to exhibit strong path dependence, long memory, and nonstationarity.
Performance Evaluation
Since perfect prediction is unattainable owing to intrinsic uncertainty, the ultimate aim of performance evaluation must be redefined. With the Balanced Scorecard, Kaplan and Norton (1992) sought to broaden the organization’s dimensions of observation, which in mathematical terms amounts to an expansion of the driving information set.
Evaluation depends on the information set 𝒢 available to the organization. Mathematically, 𝒢 is defined as a sub-σ-algebra of ℱ (that is, 𝒢 ⊂ ℱ). The structure of 𝒢 represents the “resolution” of the organization’s information system, or the manager’s “cognitive boundary,” and any metric employed in management accounting must be measurable with respect to 𝒢. In their study of subjective performance measurement, Baker et al. (1994) noted that the absence of explicit indicators must often be remedied by extending the set of subjectively observable information. For evaluation to be effective, the information set 𝒢 must capture the process data of knowledge activity, characterize task and organizational context in fine detail, and represent the interaction effects between capability and context.
Under the dual constraints of incomplete information (𝒢 ⊂ ℱ) and inherent system randomness, the core of evaluation lies in finding an optimal estimator that minimizes the mean squared error (MSE). The mean-squared-error function is defined as:
By the foundational theorems of probability theory and modern econometrics (Hayashi, 2000; Wooldridge, 2010; Greene, 2013), among all estimators based on the information set 𝒢—that is, all 𝒢-measurable functions Ŷ(𝒢)—the conditional expectation E[X | 𝒢] is the unique optimal estimator that minimizes mean squared error. This proposition may be stated as:
Decomposing the MSE of an arbitrary estimator Ŷ (using the law of iterated expectations, by which the cross term can be shown to vanish) yields an intuitive understanding of its deeper meaning:
- First term (irreducible error): E[(X − E[X | 𝒢])2] is the conditional variance Var(X | 𝒢). It represents the intrinsic randomness that remains even after the information 𝒢 is given. This is error that no estimator can eliminate, and it sets the theoretical ceiling on evaluation precision.
- Second term (reducible error): E[(E[X | 𝒢] − Ŷ)2] measures the gap between the existing evaluation instrument Ŷ and the theoretically optimal solution E[X | 𝒢].
To minimize the overall MSE, the only feasible route is to drive the second term to zero—that is, to require Ŷ = E[X | 𝒢].
Theoretical Synthesis
The foregoing argument yields a far-reaching reconstruction of the performance-evaluation paradigm. It establishes a universal and unified mathematical target for all complex evaluation systems, identifying E[X | 𝒢] as the optimal evaluation result under a given information set. This shift marks the elevation of performance management from an art reliant on experiential intuition to a science with a well-defined objective function that can be quantitatively verified. The design of every performance-evaluation mechanism can therefore be reduced to a single unified proposition: how to construct and exploit the information set 𝒢 more efficiently, so as to approximate the conditional expectation E[X | 𝒢] with greater precision. The remainder of this paper puts that proposition to an empirical test.
Research Hypotheses
Once performance evaluation is recast as the problem of solving for the conditional expectation E[X | 𝒢], the failure of traditional evaluation tools can be traced to a single root cause: their model complexity cannot match the complexity of the real world, which produces systematic high bias. Improving estimation accuracy therefore hinges on constructing new estimators capable of processing higher-dimensional, more richly structured information sets 𝒢. On this basis, we advance three core hypotheses.
- H1 (complexity-matching hypothesis). An estimator’s predictive validity—its accuracy in approximating E[X | 𝒢]—is positively associated with the complexity (dimensionality and degree of structure) of the information set 𝒢 it can process.
- H2 (new-paradigm superiority hypothesis). Because it makes effective use of high-dimensional structured information—such as the personalized capability–task matrix (PCTM) and the organizational knowledge base (OKB)—the novel computational estimator proposed here (DEPICT) will exhibit predictive validity significantly superior to that of traditional and existing benchmark estimators.
- H3 (diagnostic-validity hypothesis). The DEPICT framework follows a “glass-box” design principle, so that it can deliver accurate predictions while also providing the interpretability and diagnostic validity that management accounting requires.
Research Design
Knowledge-work data drawn from real corporate environments routinely present insurmountable challenges: difficulty of access, ambiguity in the performance “ground truth,” and contamination by confounding variables. To ensure that the core mechanism can be isolated and studied cleanly, this study adopts the dataset of a large-scale online platform for professional accounting education as its “natural laboratory.” Accounting and law are disciplines with highly structured knowledge systems, which furnish a blueprint for constructing an objective organizational knowledge base (OKB); at the same time, standardized response outcomes provide an unbiased “ground truth” for measuring the performance of a single instance of knowledge application.
The study operationalizes the theoretical variables as the following measurable indicators.
Target variable (true performance)
X is operationalized as an individual’s response to a particular item j at a particular point in time t:
where 0 denotes an incorrect answer and 1 denotes a correct answer.
Information set (observable information space)
𝒢t is defined as all information available prior to the prediction instant t, specifically encompassing individual identity, task-content information, the historical response sequence, and structured knowledge information (the OKB knowledge graph).
Estimator (performance-evaluation instrument)
Ê[X | 𝒢] denotes a series of concrete computational models whose output is an estimate of the probability of performance at the next instant.
Model Specification
The experiment uses two datasets—Basic Accounting Practice (BAP) and Economic Law (EL)—and constructs a structured knowledge graph spanning content, hierarchy, and relational links. To evaluate efficacy systematically, we build an “estimator-capability ladder” comprising benchmark models with differing levels of information-processing power.
Table 1 Information-set specifications: benchmark and novel estimators
| Estimator | Theoretical paradigm | Information set 𝒢 | Temporal dynamics | Model complexity |
|---|---|---|---|---|
| IRT | Psychometrics | Individual and task identifiers only | No | Low |
| DNN | Static, data-driven | Individual identifier and task text content | No | Medium |
| DKT | Dynamic, data-driven | Individual identifier and historical response sequence | Yes | High |
| EERNN | Dynamic, data-driven (enhanced) | Individual identifier, historical sequence, and text content | Yes | High |
| DEPICT | Computational management accounting | Individual identifier, historical sequence, text, and structured OKB | Yes | Very high |
To quantify precisely the value of the different types of information in the structured knowledge base, we designed an ablation experiment on the full DEPICT model.
- DEPICT-T (no structural information). Uses only the textual descriptions of knowledge points, isolating the value of “knowledge content.”
- DEPICT-H (flattened structure). Adds the hierarchical relations among knowledge points, isolating the value of local “knowledge-hierarchy structure.”
- DEPICT-P (global context path). Adds global path information, isolating the value of “global knowledge context.”
Results and Discussion
The results show that on both the BAP and EL datasets the DEPICT model attains the best predictive validity (see Table 2), outperforming all benchmark estimators comprehensively and significantly (p < 0.001). This strongly supports H2 and demonstrates that explicitly incorporating structured knowledge information is decisive for improving the accuracy of performance evaluation.
Table 2 Comparative analysis of overall predictive validity (illustrative data)
| Estimator | BAP — AUC | BAP — ACC | EL — AUC | EL — ACC |
|---|---|---|---|---|
| IRT | 0.710 | 0.650 | 0.705 | 0.645 |
| DNN | 0.750 | 0.680 | 0.745 | 0.675 |
| DKT | 0.780 | 0.710 | 0.775 | 0.705 |
| EERNN | 0.825 | 0.755 | 0.820 | 0.750 |
| DEPICT | 0.850 | 0.780 | 0.845 | 0.775 |
A horizontal comparison of the data clearly reveals a stepwise pattern in which estimator efficacy rises systematically with information-processing capacity, validating H1.
- First-order transition (content information). Models that incorporate task content (such as DNN and EERNN) significantly outperform models that ignore it, indicating that the concrete task context is essential to prediction.
- Second-order transition (dynamic information). Dynamic sequence models generally outperform static ones, confirming the necessity of capturing the evolution of capability over time (the learning effect).
- Third-order transition (structured information). DEPICT breaks through the performance ceiling of EERNN, demonstrating the necessity of matching information complexity.
The ablation experiment quantifies the incremental contribution of knowledge information and reveals a clear hierarchy (see Table 3). Refined “knowledge content” supplies a high-quality semantic signal; “knowledge structure (hierarchy)” helps the model grasp logical relations and levels of abstraction; and “global context” further captures complex knowledge-integration effects. This systematically confirms that theory-driven structuring of the information set is the core pathway to reducing estimation bias.
Table 3 Ablation results: the value hierarchy of knowledge information
| Model variant | Information dimension added | AUC | Performance gain | Value tier |
|---|---|---|---|---|
| EERNN | Content + sequence | 0.825 | — | — |
| DEPICT-T | + knowledge content | 0.830 | 0.005 | Tier 1: content value |
| DEPICT-H | + knowledge structure | 0.840 | 0.010 | Tier 2: structural value |
| DEPICT | + knowledge context | 0.850 | 0.010 | Tier 3: contextual value |
Conclusion
Through a comparative experiment, this study is the first to quantify empirically the cost of “estimator failure”—the price that traditional management-accounting tools pay for their model simplicity and information loss. The marked advantage of dynamic models over static ones provides quantitative support for the theoretical notion of “capability in action,” confirming that performance arises from the dynamic interaction of capability with task context.
More importantly, the study demonstrates that injecting management theory into computational models does not impair accuracy; on the contrary, by supplying a beneficial inductive bias, it breaks through the performance ceiling of purely data-driven models. Although research conducted in a “natural laboratory” still has limitations regarding generalizability to corporate environments, the empirical findings here lay a solid foundation for exploring a performance-evaluation paradigm that deeply integrates theory and computation.
References
Arrow, K. J. (1962). The economic implications of learning by doing. The Review of Economic Studies, 29(3), 155–173.
Baker, G., Gibbons, R., & Murphy, K. J. (1994). Subjective performance measures in optimal incentive contracts. The Quarterly Journal of Economics, 109(4), 1125–1156.
Demski, J. S. (1976). Uncertainty and evaluation based on controllable performance. The Accounting Review, 51(2), 230–245.
Greene, W. H. (2013). Econometric analysis (7th ed.). Pearson Education.
Hayashi, F. (2000). Econometrics. Princeton University Press.
Holmström, B. (1979). Moral hazard and observability. The Bell Journal of Economics, 10(1), 74–91.
Ittner, C. D., & Larcker, D. F. (2003). Coming up short on nonfinancial performance measurement. Harvard Business Review, 81(11), 88–95.
Kaplan, R. S., & Norton, D. P. (1992). The Balanced Scorecard—measures that drive performance. Harvard Business Review, 70(1), 71–79.
Teece, D. J., Pisano, G., & Shuen, A. (1997). Dynamic capabilities and strategic management. Strategic Management Journal, 18(7), 509–533.
Wooldridge, J. M. (2010). Econometric analysis of cross section and panel data (2nd ed.). MIT Press.