Tools for MLN evaluation

PRACMLNs comes with a set of tools for convenient evaluation of an MLN or learning/inference algorithms.

Confusion Matrices

Examples

>>> cm = ConfusionMatrix()
>>> for _ in range(10):
...     cm.addClassificationResult("AAA","A")
>>> cm.addClassificationResult("AAA","AAA")
>>> cm.addClassificationResult("AAA","AAA")
>>> cm.addClassificationResult("AAA","AAA")
>>> cm.addClassificationResult("AAA","AAA")
>>> cm.addClassificationResult("AAA","B")
>>> cm.addClassificationResult("AAA","B")
>>> cm.addClassificationResult("AAA","C")
>>> cm.addClassificationResult("B","AAA")
>>> cm.addClassificationResult("B","AAA")
>>> cm.addClassificationResult("B","C")
>>> cm.addClassificationResult("B","B")
>>> cm.printTable()
-------------------------------
| P\C |   A | AAA |   B |   C |
|-----+-----+-----+-----+-----|
|   A |   0 |   0 |   0 |   0 |
|-----+-----+-----+-----+-----|
| AAA |  10 |   4 |   2 |   1 |
|-----+-----+-----+-----+-----|
|   B |   0 |   2 |   1 |   1 |
|-----+-----+-----+-----+-----|
|   C |   0 |   0 |   0 |   0 |
-------------------------------

>>> cm.printPrecisions()
A: - Acc=0.52, Pre=0.00, Rec=0.00 F1=0.00

AAA: - Acc=0.29, Pre=0.24, Rec=0.67 F1=0.35

B: - Acc=0.76, Pre=0.25, Rec=0.33 F1=0.29

C: - Acc=0.90, Pre=0.00, Rec=0.00 F1=0.00

>>> cm.getLatexTable()
\footnotesize
\begin{tabular}{|l|l|l|l|l|}
\hline
Prediction/Ground Truth & \begin{turn}{90}A\end{turn} & \begin{turn}{90}AAA\end{turn} & \begin{turn}{90}B\end{turn} & \begin{turn}{90}C\end{turn}\\ \hline
A & \cellcolor{cfmcolor!0}\textbf{0} & \cellcolor{cfmcolor!0}0 & \cellcolor{cfmcolor!0}0 & \cellcolor{cfmcolor!0}0\\ \hline
AAA & \cellcolor{cfmcolor!53}10 & \cellcolor{cfmcolor!21}\textbf{4} & \cellcolor{cfmcolor!11}2 & \cellcolor{cfmcolor!5}1\\ \hline
B & \cellcolor{cfmcolor!0}0 & \cellcolor{cfmcolor!33}2 & \cellcolor{cfmcolor!17}\textbf{1} & \cellcolor{cfmcolor!17}1\\ \hline
C & \cellcolor{cfmcolor!0}0 & \cellcolor{cfmcolor!0}0 & \cellcolor{cfmcolor!0}0 & \cellcolor{cfmcolor!0}\textbf{0}\\ \hline
\end{tabular}

>>> cm.getPDF('example.pdf')

The last command will produce a PDF example.pdf, which will approzimately look like the following:

_images/conf_matrix.png