Diagnostics
Every diagnostic returns a list of Finding records. A finding has an id
(<diagnostic>:<subject>, for example crux:strength:r2), the ids of the
nodes and relations involved, a headline value, the numbers behind it in
details, and a one-line message. Finding.to_dict() gives its JSON form.
| Function | Reports |
|---|---|
missing_parameters |
An inference variable without a base |
unanchored |
An inference variable with no source and no inferential parent |
correlated_support |
Supports of a node that share a source and no common ancestor, or one variable |
claims |
|logit(stated) − logit(computed)| above a threshold (default 2 ln(11/9) ≈ 0.40) |
sensitivity |
∂P(T)/∂θ for every parameter θ |
crux |
|∂P(T)/∂θ| · sd(θ): T depends on θ and θ is uncertain |
single_points_of_failure |
Y with P(T | do(Y = false)) below a fraction of P(T) (0.1) |
failure_impact |
Every Y ranked by P(T | do(Y = false)) / P(T), the threshold marking the line |
value_of_information |
The mutual information I(T; Y) in bits |
When no parameter T depends on is uncertain, as in a graph of plain numbers,
every crux is zero; crux then returns a single finding crux:<target>, with
no value, that says so instead of ranking the zeros. Whether T depends on a
parameter is decided from the network, not from the size of a derivative, so
every engine gives the same answer: θ does not count when it is d-separated from
T given the evidence, or when it cancels out of its own table, in the rows the
evidence leaves, in exact arithmetic (a requires strength whose premise is
observed true, the base of an observed-true child whose supports are all
observed, a base multiplied by a requires strength of exactly 1 whose premise
is observed false). A derivative that is zero only because contributions through
different tables cancel is not detected, and θ is then still ranked. Nor is one
that is zero only because θ becomes a common factor once a free parent of its
table, neither observed nor T, is summed out: the rows of the table alone do not
show it. With T refuting an observed-true X and a free root M requiring X, the
requires strength multiplies both values of T alike once M is summed out, yet
it is still ranked; observing M as well lets the table show it.
The cancellation check enumerates the rows of the carrier's table that the
evidence leaves, in exact rational arithmetic, so its worst-case cost grows like
2^k, where k is the number of the carrier's parents not fixed by evidence,
interventions or exclusive constraints; that parent count is the scaling
limit. The check returns as soon as it reaches a row that rules cancellation
out, so it can stop before the last row, and crux checks only uncertain
parameters and stops at the first one T depends on. A parameter that does cancel
always costs the full enumeration.
diagnose(graph, target) runs them all. If a parameter is missing the graph
cannot be compiled, and the report stops after the structural checks.
Under evidence¶
Every inference diagnostic, and diagnose, takes evidence, a mapping of node
ids to observed values, written E below. Without it, or with {}, each answers
for the graph before anything is observed, exactly as before evidence existed.
With it:
| Diagnostic | Under evidence E |
|---|---|
claims |
stated against P(X | E); observed variables are skipped |
sensitivity |
∂P(T | E)/∂θ, by the quotient rule below |
crux |
|∂P(T | E)/∂θ| · sd(θ), sd(θ) of the parameter's own credence |
single_points_of_failure |
Y with P(T | do(Y = false), E) below a fraction of P(T | E) |
failure_impact |
P(T | do(Y = false), E) / P(T | E), over the same candidates |
value_of_information |
I(T; Y | E) in bits, over the unobserved variables Y |
The target may not be observed, directly or through an equivalent node, since
its diagnostics would all be trivial; that is a ValidationError. Evidence of
probability zero, including equivalent nodes observed at different values, is a
ZeroProbabilityError. A single point of failure is not reported for a variable
whose failure the evidence rules out, P(E | do(Y = false)) = 0: given what was
observed, Y did not fail. Crux keeps sd(θ) from the parameter's credence as
written; the parameters are not updated on the evidence, as they are not for the
uncertainty bands of a query.
Sensitivity under evidence is not a two-point difference. Write C for the
exclusive constraints, observed in every query, and
N(θ) = P(T, E, C) and D(θ) = P(E, C),
so that P(T | E) = N(θ) / D(θ). Each of N and D is a joint probability of the network, so each is a straight line in θ: N(θ) = N(0) + θ N′ with N′ = N(1) − N(0), and likewise D. Their ratio is not a line, so P(T | E, θ = 1) − P(T | E, θ = 0) is not its derivative; the quotient rule is exact instead:
∂P(T | E)/∂θ = (N′ D(θ) − N(θ) D′) / D(θ)²,
from N and D at θ = 0, at θ = 1, and at the value θ holds. The fast two-point
identity ∂P(T)/∂θ = P(T | θ = 1) − P(T | θ = 0) may be used only when D does not
depend on θ: with no evidence and no exclusive constraint, where D = 1. For
one observation o of a hypothesis H with base h, P(o | H) = L and P(o | not H) =
b, P(H | o) = hL / (hL + (1 − h) b) runs from 0 at h = 0 to 1 at h = 1, a
difference of 1, while its derivative at h = 0.3, L = 0.8, b = 0.2 is Lb / (hL +
(1 − h) b)² = 1.108.
Sensitivity is exact. Every joint probability of the network is multilinear
in the parameters, so without exclusive constraints or evidence P(T) is a
straight line in each θ and ∂P(T)/∂θ = P(T | θ = 1) − P(T | θ = 0). An
exclusive constraint C is observed in every query, which makes P(T) = P(T, C)
/ P(C) a ratio of two such lines; the two-point difference is then not the
derivative, and the quotient rule is used instead, as under evidence above.
Claims are compared in log-odds but reported in probability. A claims
finding's value is the signed gap stated − computed, and its details hold
stated and computed, all probabilities; the threshold it was tested against
is in natural log-odds and is named for its unit, threshold_log_odds, so the
two scales cannot be confused in the JSON report.
Crux is the primary weak-point ranking. A steep derivative on a parameter known exactly, or a wide credence the target ignores, both score zero.
Single points of failure use intervention, not conditioning: "suppose this premise is simply wrong". The threshold is a fraction of the target's own probability, not a probability: by default a variable is reported when its failure leaves P(T) below a tenth of what it was. An already improbable target therefore still has its premises ranked, rather than every premise whose failure lowers it reported; a threshold of 1 reports all of those.
A premise behind one requires of strength r crosses the threshold only if r
> 1 − threshold. Suppose the only directed path from Y to T is one requires
relation Y → T of strength r, with no evidence and no exclusive constraint.
P(T = 1 | parents) is Y's necessity gate, 1 − r when Y is false and 1 when it is
true, times a factor A that does not involve Y. Failing Y leaves T's other
parents as they were, so P(T | do(Y = false)) = (1 − r) E[A], while P(T) ≤ E[A]
because the gate is at most 1:
P(T | do(Y = false)) / P(T) ≥ 1 − r.
When Y shares no ancestor with T's other parents the ratio is exactly (1 − r) /
((1 − r) + r p), with p = P(Y). At the default threshold of 0.1 such a premise
is never a single point of failure when r ≤ 0.9, however probable it is, so an
empty list says that no requires is stronger than 0.9, not that nothing is
fatal.
Failure impact ranks what lies below the line. failure_impact gives, for
every candidate of single_points_of_failure, the ratio P(T | do(Y = false), E)
/ P(T | E) as its value, lowest first, with P(T | E) as baseline, P(T | do(Y
= false), E) as p_target_if_false and the threshold in its details. The
threshold does not filter it: single_point_of_failure is true exactly for the
variables single_points_of_failure reports. The premises can therefore be
ranked by how much their failure dents the target without rerunning at another
threshold. When P(T | E) is zero the ratio is undefined and there are no
findings.
Parameters are treated as independent, so the most common source of overconfidence in a hand-built argument is a missing common cause: two premises that share an unstated assumption. Model the assumption as a proposition of its own with a relation into each premise, and it will show up in these rankings.
Correlated supports are the same gap in the arguments themselves. The
supports relations into a node are a noisy OR of independent reasons, so ten
of strength 0.2 into a node of base 0 give it 1 − 0.8¹⁰ = 0.893, whether they
are ten experiments or one source's reasoning written down ten times.
correlated_support reports, for each node, a group of at least min_supports
(2 by default) supporting variables that share a source, anchored in the same
document or pointing at the same node through an authored_by or derived_from
annotation, when no variable is a common ancestor of them through relations of
nonzero strength: an ancestor of every one of them, or one of them that is an
ancestor of all the others, as when p0 supports both p1 and the node and
p1 supports the node. A variable with no parents and a base of mean exactly 0
or 1 is not a common ancestor: it never varies, so it correlates nothing. Any
other common ancestor silences the finding, whatever it stands for and however
weak its relations: one broad assumption that every one of them rests on, such
as "the instrument works", hides the group even when the supports still come
from one source's reasoning. The finding has no value; its details hold
supports, the size of the group, and min_supports. The fix is a common
parent that each of them requires with strength 1: the group can then lift a
node of base b to at most b + (1 − b) c, c the common parent's credence, however
large it is. See
Reading the diagnostics
for a worked example.
Several supports relations from one variable into the same node, such as one
from each of two nodes merged as equivalent, are separate terms of the same
noisy OR: two of strength 0.2 give a node of base 0 the credence 0.36 where one
gives 0.2. correlated_support also reports at least min_supports of them of
nonzero strength, with the id correlated-support:<node>:variable:<variable>,
whatever the variable's parents: they already share their only input, so no
common parent can correlate them. The fix is one relation in their place.
The reference below has an entry for each function and class that
credencegraph.diagnostics exports. claims is listed under its own module,
credencegraph.diagnostics.claims, whose name it shares. The constants it
exports have no entries of their own. The defaults DEFAULT_CLAIM_THRESHOLD,
DEFAULT_FAILURE_THRESHOLD and DEFAULT_MIN_SUPPORTS appear by name in the
signatures and parameter tables of the functions that take them; the finding
kinds, such as OVERCLAIM, and PROVENANCE_TYPES appear in no signature or
parameter table. The module credencegraph.diagnostics.weak_points also defines
Interventions, interventions, sensitivity_findings, crux_findings,
single_point_of_failure_findings and failure_impact_findings, which
diagnose and diagnose_many use to build their reports. They are internal to
the diagnostics package, not exported by credencegraph.diagnostics and not
part of its public API, so they are not documented here; call the diagnostics
above instead.
Diagnostics: structural checks, claim consistency, weak points, discharges and sorries.
Structural checks, stated-versus-computed consistency and the weak points of a target each return
a list of Finding records, each with an id, the ids of the nodes and relations involved, a
headline value and a one-line explanation; Finding.to_dict gives its JSON form. verify
instead returns one Check per discharge, holding the findings of its re-check, and sorries
returns a SorryReport of the steps a target depends on that rest on judgement only.
Check
dataclass
¶
The re-check of one discharge.
Attributes:
| Name | Type | Description |
|---|---|---|
relation |
str
|
The id of the relation whose obligation the discharge meets. |
kind |
str
|
The discharge's kind. |
outcome |
str
|
|
findings |
tuple[Finding, ...]
|
What the re-check found wrong. |
status |
int | None
|
The exit status of the script's run now, |
reason |
str | None
|
Why an unchecked discharge was not checked. |
Source code in src/credencegraph/diagnostics/obligations.py
187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 | |
to_dict()
¶
Return the check as JSON-ready data.
Returns:
| Type | Description |
|---|---|
dict[str, JSON]
|
|
Source code in src/credencegraph/diagnostics/obligations.py
210 211 212 213 214 215 216 217 218 219 220 221 222 223 | |
Finding
dataclass
¶
One result of a diagnostic.
Attributes:
| Name | Type | Description |
|---|---|---|
id |
str
|
Identifies the finding within a report: |
diagnostic |
str
|
The diagnostic that produced it, such as |
message |
str
|
A one-line explanation for a human reader. |
nodes |
tuple[str, ...]
|
The ids of the nodes involved. |
relations |
tuple[str, ...]
|
The ids of the relations involved. |
value |
float | None
|
The diagnostic's headline number, or |
details |
Mapping[str, float]
|
The other numbers behind the finding, by name. |
Source code in src/credencegraph/diagnostics/records.py
30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | |
__post_init__()
¶
Store read-only copies of the id tuples and the details.
Source code in src/credencegraph/diagnostics/records.py
54 55 56 57 58 | |
to_dict()
¶
Return the finding as JSON-ready data.
Returns:
| Type | Description |
|---|---|
dict[str, JSON]
|
|
Source code in src/credencegraph/diagnostics/records.py
60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | |
Report
dataclass
¶
The findings for a graph and for each of several target nodes.
Attributes:
| Name | Type | Description |
|---|---|---|
findings |
tuple[Finding, ...]
|
The graph-wide findings: missing parameters, unanchored variables, correlated supports, overclaims and underclaims. They do not depend on a target and appear once. |
weak_points |
Mapping[str, tuple[Finding, ...]]
|
Each target's sensitivity, crux, single points of failure, failure impact and
value of information, by target id in the order the targets were given. A finding id is
unique within one target's list; the same id, such as |
Source code in src/credencegraph/diagnostics/report.py
30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | |
__post_init__()
¶
Store read-only copies of the findings and of the weak points.
Source code in src/credencegraph/diagnostics/report.py
46 47 48 49 50 51 52 53 | |
all_findings()
¶
Return the graph-wide findings, then each target's.
Returns:
| Type | Description |
|---|---|
list[Finding]
|
Every finding, in report order. |
Source code in src/credencegraph/diagnostics/report.py
55 56 57 58 59 60 61 | |
to_dicts()
¶
Return every finding as JSON-ready data, naming the target of each.
Returns:
| Name | Type | Description |
|---|---|---|
list[dict[str, JSON]]
|
The graph-wide findings, then each target's, as |
|
key |
list[dict[str, JSON]]
|
|
Source code in src/credencegraph/diagnostics/report.py
63 64 65 66 67 68 69 70 71 72 73 | |
Sorry
dataclass
¶
A step a target depends on that rests on judgement only.
Attributes:
| Name | Type | Description |
|---|---|---|
relation |
str
|
The relation's id. |
type |
str
|
The relation's type. |
source |
str
|
Its source node. |
target |
str
|
Its target node. |
reason |
str
|
Why it counts as a sorry, one of |
obligation |
str | None
|
The obligation's statement, if the relation has one. |
sensitivity |
float | None
|
|
Source code in src/credencegraph/diagnostics/obligations.py
387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 | |
to_dict()
¶
Return the sorry as JSON-ready data.
Returns:
| Type | Description |
|---|---|
dict[str, JSON]
|
|
Source code in src/credencegraph/diagnostics/obligations.py
409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 | |
SorryReport
dataclass
¶
The steps a target depends on, split into the discharged ones and the sorries.
Attributes:
| Name | Type | Description |
|---|---|---|
target |
str
|
The target node. |
steps |
tuple[str, ...]
|
The ids of the inferential relations the target depends on, in the graph's order. |
discharged |
tuple[str, ...]
|
The ids of those relations whose obligation is discharged by more than judgement. |
sorries |
tuple[Sorry, ...]
|
The rest, by decreasing absolute sensitivity, those without a strength last. |
Source code in src/credencegraph/diagnostics/obligations.py
426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 | |
to_dict()
¶
Return the report as JSON-ready data.
Returns:
| Type | Description |
|---|---|
dict[str, JSON]
|
|
Source code in src/credencegraph/diagnostics/obligations.py
442 443 444 445 446 447 448 449 450 451 452 453 | |
can_fail(source)
¶
Tell whether a Python script has a reachable way to fail: an assert, a raise, or a non-zero exit.
Code that a constant condition makes unreachable is ignored first: the body of if False:,
if 0: or while False:, the else of if True:, the dead branch of a conditional
expression, and an assert of a constantly true condition such as assert True. A condition
counts as constant when it is a literal, the not of a constant condition, or a conditional
expression whose condition and taken branch are constant, at any depth of nesting, so
if (False if True else False): is pruned like if False: and
assert (True if True else False) cannot fail; a check behind any other condition is taken to be
reachable. A script that does not parse has no checked failure path, only a crash, and counts as
unable to fail.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
source
|
str
|
The script's text. |
required |
Returns:
| Type | Description |
|---|---|
bool
|
|
bool
|
( |
Source code in src/credencegraph/diagnostics/obligations.py
157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 | |
correlated_support(graph, min_supports=DEFAULT_MIN_SUPPORTS)
¶
Find supports of one node that share a source but no common ancestor.
The supports relations into a node combine as a noisy OR of independent reasons: ten of
strength 0.2 into a node of base 0 give it 1 - 0.8^10, about 0.893. Reasons that come from one
source are rarely independent, and nothing in the graph says so unless they share an ancestor. Two
supporting variables share a source when a node of each is anchored in the same document, or
points at the same node through an authored_by or derived_from annotation. A group of at
least min_supports supporting variables of one node that share a source is reported when no
variable is a common ancestor of them: an ancestor, through chains of requires, supports
or refutes relations, of every one of them, or one of them that is an ancestor of all the
others, as in p0 supports p1, p0 supports claim, p1 supports claim. Either way the
graph already makes them depend on one variable, so inference does not count them as independent.
A relation of strength exactly 0 does not make a parent, nor a support: it leaves the child's
probability the same whatever the parent's value. Nor is a fixed point a common ancestor, nor an
ancestor through it: a variable that always takes the same value, whether it has no parents and
a base of mean exactly 0 or 1 or has a table that gives it that value whatever its parents are,
as with a base of 1 and only supports parents, correlates nothing and passes nothing on.
The usual fix is a common parent: a proposition such as "this source's unverified reasoning
is sound", which each of them requires with strength 1, so that they fail together. A node of
base b with no other parents then gets at most b + (1 - b) c from the group, c the credence of
the proposition, however large the group.
Nodes merged by equivalent relations are one variable, named after the earliest of them, with
the sources of all of them. A group that shares several sources is reported once, under the first.
Several supports relations from one variable into the same node, for example one from each
of two merged nodes, are separate terms of the noisy OR too: two of strength 0.2 lift a node of
base 0 to 0.36 where one gives 0.2. At least min_supports of them of nonzero strength are
reported whatever the variable's parents, since no common parent can correlate terms that
already share their only input; the fix is one relation in their place.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
graph
|
Graph
|
The graph. |
required |
min_supports
|
int
|
The smallest group reported, at least 2; the default, 2, reports any two. |
DEFAULT_MIN_SUPPORTS
|
Returns:
| Type | Description |
|---|---|
list[Finding]
|
One finding per node and group, the nodes in insertion order, and for each node its groups |
list[Finding]
|
before its repeated supports. Its id is |
list[Finding]
|
the kind is |
list[Finding]
|
as the source for supports repeated from one variable. Its nodes are the supported node, then |
list[Finding]
|
the group or the variable; its relations are their |
list[Finding]
|
into the node. It has no value; its details are |
list[Finding]
|
|
Raises:
| Type | Description |
|---|---|
ValidationError
|
If |
Source code in src/credencegraph/diagnostics/structure.py
252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 | |
crux(network, target, *, evidence=None, engine=None)
¶
Rank the parameters by |dP(target | evidence)/dθ| * sd(θ).
A parameter is a crux when the target depends on it and it is uncertain: a steep derivative on
a Point parameter, or a wide credence the target ignores, both score zero. This is the
primary weak-point ranking. Under evidence, sd(θ) is still that of the parameter's own credence:
the parameters are not updated on the evidence, as they are not for the uncertainty bands.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
network
|
Network
|
The network. |
required |
target
|
str
|
The id of the target node. |
required |
evidence
|
Mapping[str, bool] | None
|
Node ids and their observed values; none by default. |
None
|
engine
|
Engine | None
|
The exact engine; variable elimination by default. |
None
|
Returns:
| Type | Description |
|---|---|
list[Finding]
|
One finding per parameter, largest crux first; or a single structural finding, with no |
list[Finding]
|
value, when no parameter the target depends on is uncertain, as when every one is a |
list[Finding]
|
|
Raises:
| Type | Description |
|---|---|
ValidationError
|
If |
ZeroProbabilityError
|
If the evidence and the |
Source code in src/credencegraph/diagnostics/weak_points.py
489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 | |
dependencies(graph, target)
¶
Find the inferential relations a target depends on.
These are the requires, supports and refutes relations into the target or into any
node it depends on, and the equivalent and exclusive relations that touch either; the
other end of each also becomes a node the target depends on.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
graph
|
Graph
|
The graph. |
required |
target
|
str
|
The target node's id. |
required |
Returns:
| Type | Description |
|---|---|
list[Relation]
|
The relations, in the graph's order. |
Source code in src/credencegraph/diagnostics/obligations.py
456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 | |
derivatives(network, target, *, evidence=None, engine=None)
¶
Compute dP(target | evidence)/dθ exactly for every parameter θ of the network.
See the module documentation for why this is exact. Each derivative is taken at the value the parameter currently holds, with every other parameter held at its own. With evidence the derivative is the quotient rule on two multilinear evaluations, not the two-point difference.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
network
|
Network
|
The network. |
required |
target
|
str
|
The id of the target node. |
required |
evidence
|
Mapping[str, bool] | None
|
Node ids and their observed values; none by default. |
None
|
engine
|
Engine | None
|
The exact engine; variable elimination by default. |
None
|
Returns:
| Type | Description |
|---|---|
dict[ParameterKey, float]
|
The derivative of |
dict[ParameterKey, float]
|
|
Raises:
| Type | Description |
|---|---|
ValidationError
|
If |
ZeroProbabilityError
|
If the evidence and the |
Source code in src/credencegraph/diagnostics/weak_points.py
129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 | |
diagnose(graph, target=None, *, evidence=None, claim_threshold=DEFAULT_CLAIM_THRESHOLD, failure_threshold=DEFAULT_FAILURE_THRESHOLD, min_supports=DEFAULT_MIN_SUPPORTS, engine=None)
¶
Run the diagnostics that apply to a graph and, optionally, a target node.
The structural checks (missing parameters, unanchored variables, correlated supports) always
run. If a parameter is missing the graph cannot be compiled, and the report stops there. Otherwise the stated credences
are compared with the computed ones, and, when a target is given, its sensitivity, crux,
single points of failure, failure impact and value of information follow. diagnose_many
does the same for several targets, running the graph-wide checks once.
With evidence every inference diagnostic is taken given it: the computed credences are
P(X | evidence), the sensitivities and cruxes are derivatives of P(target | evidence),
the single points of failure and the failure impact compare P(target | do(Y = false), evidence)
with P(target | evidence), and the value of information is I(target; Y | evidence). Observed
variables are left out of the comparisons and rankings over nodes, and the target may not be
observed. Without evidence the report is the one for the graph before anything is observed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
graph
|
Graph
|
The graph. |
required |
target
|
str | None
|
The id of the node whose weak points are wanted, if any. |
None
|
evidence
|
Mapping[str, bool] | None
|
Node ids and their observed values; none by default. |
None
|
claim_threshold
|
float
|
The threshold passed to |
DEFAULT_CLAIM_THRESHOLD
|
failure_threshold
|
float
|
The threshold passed to |
DEFAULT_FAILURE_THRESHOLD
|
min_supports
|
int
|
The smallest group passed to |
DEFAULT_MIN_SUPPORTS
|
engine
|
Engine | None
|
The exact engine; variable elimination by default. |
None
|
Returns:
| Type | Description |
|---|---|
list[Finding]
|
The findings, grouped by diagnostic in the order above; |
list[Finding]
|
ready for |
Raises:
| Type | Description |
|---|---|
ValidationError
|
If a threshold or |
CompileError
|
If the graph has no missing parameter but still fails to compile, for example over conflicting bases on merged nodes. |
ZeroProbabilityError
|
If the evidence and the |
Source code in src/credencegraph/diagnostics/report.py
76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 | |
diagnose_many(graph, targets, *, evidence=None, claim_threshold=DEFAULT_CLAIM_THRESHOLD, failure_threshold=DEFAULT_FAILURE_THRESHOLD, min_supports=DEFAULT_MIN_SUPPORTS, engine=None)
¶
Run the graph-wide diagnostics once and the weak-point diagnostics for each of several targets.
The graph-wide findings are those diagnose reports without a target; each target's weak
points are those diagnose adds for it, under the same evidence and thresholds. A target named
twice is diagnosed once. If a parameter is missing the graph cannot be compiled, and every
target's list is empty.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
graph
|
Graph
|
The graph. |
required |
targets
|
Iterable[str]
|
The ids of the nodes whose weak points are wanted; may be empty. |
required |
evidence
|
Mapping[str, bool] | None
|
Node ids and their observed values; none by default. |
None
|
claim_threshold
|
float
|
The threshold passed to |
DEFAULT_CLAIM_THRESHOLD
|
failure_threshold
|
float
|
The threshold passed to |
DEFAULT_FAILURE_THRESHOLD
|
min_supports
|
int
|
The smallest group passed to |
DEFAULT_MIN_SUPPORTS
|
engine
|
Engine | None
|
The exact engine; variable elimination by default. |
None
|
Returns:
| Type | Description |
|---|---|
Report
|
The graph-wide findings, and each target's weak points in the order the targets were given; |
Report
|
|
Raises:
| Type | Description |
|---|---|
ValidationError
|
If |
CompileError
|
If the graph has no missing parameter but still fails to compile, for example over conflicting bases on merged nodes. |
ZeroProbabilityError
|
If the evidence and the |
Source code in src/credencegraph/diagnostics/report.py
137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 | |
failure_impact(network, target, *, evidence=None, threshold=DEFAULT_FAILURE_THRESHOLD, engine=None)
¶
Rank the other variables Y by P(target | do(Y = false), evidence) / P(target | evidence).
The ratio is the fraction of the target's probability that survives the failure of Y, with the
intervention of single_points_of_failure: below 1 the failure lowers the target, above 1 it
raises it. Every candidate of single_points_of_failure is ranked, lowest ratio first, and the
threshold only marks the line: a finding whose ratio falls below it, under the same rounding
rule, is exactly one single_points_of_failure reports. The ranking therefore orders the
premises by failure impact without a second run at another threshold, which matters because a
premise behind one requires of strength r keeps at least 1 - r of the target and crosses
the default line only for r > 0.9; see single_points_of_failure.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
network
|
Network
|
The network. |
required |
target
|
str
|
The id of the target node. |
required |
evidence
|
Mapping[str, bool] | None
|
Node ids and their observed values; none by default. |
None
|
threshold
|
float
|
The fraction of |
DEFAULT_FAILURE_THRESHOLD
|
engine
|
Engine | None
|
The exact engine; variable elimination by default. |
None
|
Returns:
| Type | Description |
|---|---|
list[Finding]
|
One finding per candidate, lowest ratio first, ties in network order; its value is the ratio, |
list[Finding]
|
and its details give |
list[Finding]
|
|
list[Finding]
|
|
list[Finding]
|
|
list[Finding]
|
there are no findings. |
Raises:
| Type | Description |
|---|---|
ValidationError
|
If |
ZeroProbabilityError
|
If the evidence and the |
Source code in src/credencegraph/diagnostics/weak_points.py
752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 | |
missing_parameters(graph)
¶
Find inference variables without a base and inferential relations without a strength.
Nodes merged by equivalent relations form one variable, which needs a base on at least one of
them. A graph with any such finding cannot be compiled: there is no default credence.
Relation already refuses to be built without a strength when its type needs one, so the
second check only fires on a graph whose relations were altered after validation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
graph
|
Graph
|
The graph. |
required |
Returns:
| Type | Description |
|---|---|
list[Finding]
|
One finding per variable without a base, then one per relation without a strength. |
Source code in src/credencegraph/diagnostics/structure.py
38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 | |
sensitivity(network, target, *, evidence=None, engine=None)
¶
Report dP(target | evidence)/dθ for every parameter θ.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
network
|
Network
|
The network. |
required |
target
|
str
|
The id of the target node. |
required |
evidence
|
Mapping[str, bool] | None
|
Node ids and their observed values; none by default. |
None
|
engine
|
Engine | None
|
The exact engine; variable elimination by default. |
None
|
Returns:
| Type | Description |
|---|---|
list[Finding]
|
One finding per parameter, largest magnitude first; its value is the signed derivative. |
Raises:
| Type | Description |
|---|---|
ValidationError
|
If |
ZeroProbabilityError
|
If the evidence and the |
Source code in src/credencegraph/diagnostics/weak_points.py
462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 | |
single_points_of_failure(network, target, *, evidence=None, threshold=DEFAULT_FAILURE_THRESHOLD, engine=None)
¶
Find the variables Y whose failure alone sinks the target: P(target | do(Y = false), evidence).
The intervention cuts the relations into Y and fixes it false, as in "suppose this premise is
simply wrong"; the evidence is then conditioned on. A variable is reported when that probability
is below threshold * P(target | evidence): the threshold is a fraction of the target's own
probability, not a probability, so the same default separates the premises that sink the target
from those that merely dent it whether the target starts at 0.9 or at 0.01. A threshold of 1
reports every variable whose failure lowers the target at all; 0 reports none.
A premise behind one requires of strength r is reported only if r > 1 - threshold. Let
the only directed path from Y to T be one requires relation Y -> T of strength r, with no
evidence and no exclusive constraint. P(T = 1 | parents) is Y's necessity gate, 1 - r
when Y is false and 1 when it is true, times a factor A that does not involve Y. Failing Y leaves
the distribution of T's other parents as it was, so P(T | do(Y = false)) = (1 - r) E[A],
while P(T) <= E[A] because the gate is at most 1. Hence
P(T | do(Y = false)) / P(T) >= 1 - r,
and when Y shares no ancestor with T's other parents the ratio is exactly
(1 - r) / ((1 - r) + r p) with p = P(Y), which tends to 1 - r as p tends to 1. At the
default threshold of 0.1 such a premise is never reported when r <= 0.9, however probable it is:
an empty list then says that no requires is stronger than 0.9, not that nothing is fatal.
failure_impact ranks every candidate by the same ratio, so the premises that dent the target
most are visible whether or not one crosses the line.
Observed variables are not candidates: the evidence already says whether they hold. Nor is a
variable whose failure the evidence rules out, where P(evidence | do(Y = false)) is zero:
given what was observed, Y did not fail.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
network
|
Network
|
The network. |
required |
target
|
str
|
The id of the target node. |
required |
evidence
|
Mapping[str, bool] | None
|
Node ids and their observed values; none by default. |
None
|
threshold
|
float
|
The fraction of |
DEFAULT_FAILURE_THRESHOLD
|
engine
|
Engine | None
|
The exact engine; variable elimination by default. |
None
|
Returns:
| Type | Description |
|---|---|
list[Finding]
|
One finding per single point of failure, lowest remaining probability first; its value is |
list[Finding]
|
|
list[Finding]
|
|
Raises:
| Type | Description |
|---|---|
ValidationError
|
If |
ZeroProbabilityError
|
If the evidence and the |
Source code in src/credencegraph/diagnostics/weak_points.py
689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 | |
sorries(graph, target, directory, *, engine=None)
¶
Report the steps a target depends on that remain sorries, ranked by the target's sensitivity.
Each sorry with a strength is ranked by dP(target)/d strength, taken exactly as in
derivatives with every parameter at its credence mean. A discharged step is not ranked:
credence applies only to what remains unproven.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
graph
|
Graph
|
The graph, which must compile for inference. |
required |
target
|
str
|
The target node's id. |
required |
directory
|
str | Path
|
The directory script paths are relative to. |
required |
engine
|
Engine | None
|
The exact engine; variable elimination by default. |
None
|
Returns:
| Type | Description |
|---|---|
SorryReport
|
The report. |
Raises:
| Type | Description |
|---|---|
CompileError
|
If the graph cannot be compiled. |
ValidationError
|
If |
ZeroProbabilityError
|
If no world satisfies the |
Source code in src/credencegraph/diagnostics/obligations.py
525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 | |
unanchored(graph)
¶
Find inference variables with no source and no inferential parents.
Such a variable is an assumption nobody wrote down: nothing in the graph says where its credence
comes from. Nodes merged by equivalent relations are one variable, anchored if any of them
has a source or is the target of a requires, supports or refutes relation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
graph
|
Graph
|
The graph. |
required |
Returns:
| Type | Description |
|---|---|
list[Finding]
|
One finding per unanchored variable, in insertion order. |
Source code in src/credencegraph/diagnostics/structure.py
79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 | |
value_of_information(network, target, *, evidence=None, engine=None)
¶
Rank the other variables Y by the mutual information I(target; Y | evidence), in bits.
I(T; Y | E) is the expected reduction in the entropy of T from learning whether Y is true,
once E is known, so the top of the ranking is what would most change the target if it were
resolved next, and is the first thing to investigate. It is at most 1 bit, the entropy of a
binary variable. Observed variables are left out: learning them again tells nothing.
The unit, bits (logarithms to base 2), is a choice of this package; the ranking does not depend
on it. Multiply by ln 2 for nats.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
network
|
Network
|
The network. |
required |
target
|
str
|
The id of the target node. |
required |
evidence
|
Mapping[str, bool] | None
|
Node ids and their observed values; none by default. |
None
|
engine
|
Engine | None
|
The exact engine; variable elimination by default. |
None
|
Returns:
| Type | Description |
|---|---|
list[Finding]
|
One finding per other unobserved proposition variable, most informative first. Its details |
list[Finding]
|
give |
list[Finding]
|
probability, |
Raises:
| Type | Description |
|---|---|
ValidationError
|
If |
ZeroProbabilityError
|
If the evidence and the |
Source code in src/credencegraph/diagnostics/weak_points.py
817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 | |
verify(graph, directory, *, timeout=DEFAULT_TIMEOUT)
¶
Re-check every discharge in a graph.
A script discharge is run with the current interpreter, from the script's directory, and is
flagged when the script cannot be read, was never run, changed since its recorded run, has no
reachable way to fail (see can_fail), or now fails: as contradicted when its recorded run
passed, or when it now passes though its recorded run failed, and as check-failed otherwise.
When a numerical script exits 0, the last non-blank line it printed is its result, flagged as
result-missing when it is not a number and as out-of-tolerance when it lies further than
tolerance from value. A citation is flagged when its
quote is not in the cited document, read as a file relative to directory, or its digest is
not the quote's; a citation of anything else is unchecked. A formal discharge is unchecked,
since no proof checker ships with this package.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
graph
|
Graph
|
The graph. |
required |
directory
|
str | Path
|
The directory that script paths and cited documents are relative to, normally the graph file's. |
required |
timeout
|
float
|
Seconds a script may run before it is stopped and its run counted as failed. |
DEFAULT_TIMEOUT
|
Returns:
| Type | Description |
|---|---|
list[Check]
|
One check per discharge, in the graph's relation order. |
Source code in src/credencegraph/diagnostics/obligations.py
346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 | |
credencegraph.diagnostics.claims.claims(graph, network=None, *, evidence=None, threshold=DEFAULT_CLAIM_THRESHOLD, engine=None)
¶
Compare each node's stated credence with the probability its premises give it.
The computed value is the point answer P(X | evidence) of the network, P(X) when there
is no evidence. A node whose variable is observed is skipped: its value is then the observation,
not something its premises deliver. The two are compared in
log-odds, logit(p) = ln(p / (1 - p)), so that a gap counts by the factor between the odds
rather than by the difference between the probabilities: 0.07 against 0.001 is far apart, 0.5
against 0.569 is not. When the stated mean exceeds the computed value by more than threshold
the text asserts more confidence than its own argument delivers (an overclaim); when it falls
short by more than threshold, less (an underclaim). A stated or computed value of exactly 0
or 1 is infinitely far from any other. Nodes with a stated credence that are not inference
variables have nothing to compare against and are skipped.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
graph
|
Graph
|
The graph. |
required |
network
|
Network | None
|
The network compiled from |
None
|
evidence
|
Mapping[str, bool] | None
|
Node ids and their observed values; none by default. |
None
|
threshold
|
float
|
The largest gap that is not reported, in natural log-odds: |
DEFAULT_CLAIM_THRESHOLD
|
engine
|
Engine | None
|
The inference engine; variable elimination by default. |
None
|
Returns:
| Type | Description |
|---|---|
list[Finding]
|
One finding per node whose stated and computed credences differ by more than |
list[Finding]
|
in insertion order. Its value is the signed gap in probability, stated minus computed, which |
list[Finding]
|
stays finite when the log-odds gap is not. Its details are |
list[Finding]
|
probability, and |
Raises:
| Type | Description |
|---|---|
ValidationError
|
If |
CompileError
|
If |
ZeroProbabilityError
|
If the evidence and the |
Source code in src/credencegraph/diagnostics/claims.py
67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | |