Basic formulas of SURE model evaluation
Goal structure
$$C=\bigcap_{i=1}^rB_i=\bigcap_{i=1}^r \bigcup_{j=1}^{s_i} A_{ij} $$
$B_i$ - key goal, $i=1,...,r$
$A_{ij}$ - sub goal, $i=1,...,r$, $j=1,...,s_i$
Theoretical scores
$Q(A_{ij})$ - theoretical score of $A_{ij}$ $\Rightarrow$
Score of $C$
$$Q(C)=\prod_{i=1}^rQ(B_i)
=\prod_{i=1}^r\big(1-\prod_{j=1}^{s_i}(1-Q(A_{ij})\big)
$$
Evaluation score of $C$
$$Q_e(C)=\sqrt[\Large r]{\prod_{i=1}^rQ_e(B_i)}
=\sqrt[\Large r]{\prod_{i=1}^r\bigg(1-
\sqrt[\Large s_i]{\prod_{j=1}^{s_i}(1-Q(A_{ij})}\bigg)}
$$
Empirical scores
$[x_0,x_1]$ - evaluation interval, $x_0 < x_1$.
Consider a sample of size $n$ of checklist results:
$$\quad x_{11}^{(1)},...,x_{1s_1}^{(1)},...,x_{r1}^{(1)},...,x_{rs_r}^{(1)}$$
$$...$$
$$\quad x_{11}^{(n)},...,x_{1s_1}^{(n)},...,x_{r1}^{(n)},...,x_{rs_r}^{(n)}$$
$x_{ij}^{(k)}$ - observed checklist score value for
$A_{ij}$ of $k$-th checklist data record, $k=1,...,n$
Empirical scores
for $k$-th checklist data record, $k=1,...,n$:
$$
Q^{*(k)}(A_{ij}) = q_{ij}^{*(k)}
= \frac{x_{ij}^{(k)}-x_0}{x_1-x_0}
$$
$$
Q^{*(k)}(B_i) = 1-\prod_{j=1}^{s_i}\big(1-q_{ij}^{*(k)}\big)
$$
$$
Q^{*(k)}(C) =
\prod_{i=1}^r\Big(1-\prod_{j=1}^{s_i}\big(1-q_{ij}^{*(k)}\big)\Big)
$$
Empirical scores for
total sample:
Arithmetic mean over empirical scores
for checklist data records for $k=1,...,n$.
$$
Q^*(A_{ij})=
\frac{1}{n}\sum_{k=1}^n Q^{*(k)}(A_{ij})
$$
$$
Q^*(B_i)= \frac{1}{n}\sum_{k=1}^n Q^{*(k)}(B_i)
$$
$$
Q^*(C) = \frac{1}{n}\sum_{k=1}^n Q^{*(k)}(C)
= \frac{1}{n}\sum_{k=1}^n\prod_{i=1}^r
\Big(1-\prod_{j=1}^{s_i}\big(1-q_{ij}^{*(k)}\big)\Big)
$$
Empirical evaluation scores
Empirical evaluation scores
(empirical calibrated scores)
for $k$-th checklist data record, $k=1,...,n$:
$$
Q_e^{*(k)}(A_{ij})= Q^{*(k)}(A_{ij}) = q_{ij}^{*(k)}
$$
$$
Q_e^{*(k)}(B_i)=1-\sqrt[\Large{s_i}]{
\prod_{j=1}^{s_i}\big(1-q_{ij}^{*(k)}\big)}
$$
$$
Q_e^{*(k)}(C)=
\sqrt[\Large r]{
\prod_{i=1}^r\bigg(
1-\sqrt[\Large s_i]{\prod_{j=1}^{s_i}\Big(1-q_{ij}^{*(k)}\Big)}
\bigg)
}
$$
Empirical evaluation scores for
total sample:
Arithmetic mean over empirical evaluation scores
of checklist data records for $k=1,...,n$.
$$ Q_e^*(A_{ij})
=\frac{1}{n}\sum_{k=1}^n Q_e^{*(k)}(A_{ij})
=\frac{1}{n}\sum_{k=1}^n q_{ij}^{*(k)}
=Q^*(A_{ij})
$$
$$
Q_e^*(B_i)
=\frac{1}{n}\sum_{k=1}^n Q_e^{*(k)}(B_i)
$$
$$
Q_e^*(C)= \frac{1}{n}\sum_{k=1}^nQ_e^{*(k)}(C)
=\frac{1}{n}\sum_{k=1}^n
\sqrt[\Large{r}]{
\prod_{i=1}^r\bigg(
1-\sqrt[\Large{s_i}]{\prod_{j=1}^{s_i}\Big(1-q_{ij}^{*(k)}\Big)}
\bigg)
}
$$
Some properties of empirical evaluation score:
$$
q_{11}^{(k)}=\cdots =q_{rs_r}^{(k)}=1
\quad\Rightarrow\quad
Q_e^{*(k)}(C)=1
$$
$$
Q_e^{*(k)}(B_i)=0\quad\Rightarrow\quad Q_e^{*(k)}(C)=0
$$
$$
q_{11}^{(k)}=\cdots =q_{rs_r}^{(k)}=q^*
\quad\Rightarrow\quad
Q_e^*(C)=q^*
$$
$\quad\Rightarrow\quad$ The empirical evaluation score $Q_e^*(C)$
describes an 'average' checklist score level.
190328