# Table 3 Comparison of common Level 2 BCI-Based AAC performance metrics

Metric Description/Equation Evaluation criteria
Error correction Rate enhancement Control State detection Experimental protocol Practicality
Written symbol rate (WSR) $\mathit{WSR}=\left\{\begin{array}{c}\hfill \frac{\left(2\mathit{SR}-1\right)}{c}\hfill \\ \hfill 0\hfill \end{array}\right\\begin{array}{c}\hfill ,\mathit{SR}>0.5\hfill \\ \hfill ,\mathit{else}\hfill \end{array}$
where SR = ITR × c/log2(N)

Practical bit rate (PBR) $\mathit{PBR}=\left\{\begin{array}{c}\hfill \frac{\left(2P-1\right)}{c}\hfill \\ \hfill 0\hfill \end{array}\right\×{log}_{2}\left(N\right)\begin{array}{c}\hfill ,P>0.5\hfill \\ \hfill ,\mathit{else}\hfill \end{array}$
The original reference did not report a formula, this was back-calculated from the results

Extended confusion matrix (ECM) Confusion matrix, as in Table 2, extended to allow for an extra detected class, “abstention.”
EffSYS $\mathit{Ef}{f}_{\mathit{SYS}}=\frac{1}{\overline{{L}_{\mathit{CW}}}×\overline{\mathit{ESC}}}$
Where$\overline{\mathit{ESC}}=\sum _{i=1}^{{N}_{\mathit{LA}}}\frac{\stackrel{^}{p}\left(i\right)}{2p\left(i\right)-1}$
Where LCW is the mean codeword length, NLA is the number of symbols in the logical alphabet,$\stackrel{^}{p}$ is the probability of a logical output appearing, and p(i) is the probability of identifying output i.
This equation is simplified from equation (20) of, assuming that all errors require two selections to correct. Interested readers are referred to for alternative methods of error correction.

EffSYS $\mathit{Ef}{f}_{\mathit{SYS}}\text{'}=\frac{1}{\overline{{L}_{\mathit{CW}}}×\overline{\mathit{ESC}}}$
Where$\mathit{ESC}=\sum _{i=1}^{{N}_{\mathit{LA}}}\stackrel{^}{p}×\left(\frac{1}{p\left(i\right)}+\left(\frac{1}{p\left(i\right)}-1\right)\left(\frac{1}{2*{p}_{B}-1}\right)\right)$
Where LCW is the mean codeword length, NLA is the number of symbols in the logical alphabet, and$\stackrel{^}{p}$ is the probability of a logical output appearing, as per. The symbols p(i) and pb are the probabilities of correctly identifying output i and the backspace option, respectively.

Output characters per minute (OCM) $\mathit{OCM}=\frac{\mathit{Correct}\phantom{\rule{0.5em}{0ex}}\mathit{Characters}}{\mathit{Time}\phantom{\rule{0.5em}{0ex}}\mathit{Taken}}$
BCI-Utility metric $U=\frac{E\left(\mathit{benefit}/\mathit{selection}\right)}{E\left(\mathit{time}/\mathit{selection}\right)}$
U must be derived for particular problems and can take different forms, several of which are presented in. Several of these forms are redundant with other level 2 metrics, for example, in the experiment presented in, this metric becomes identical to practical bit rate.
*
1. c: time per selection; N: number of choices; ITR: information transfer rate; P: probability of correct selection.
2. *: Not currently practical, but can be with future work.
3. Check marks indicate that the metric fulfills the evaluation criterion. 