Skip to content

Diagnostics

Every diagnostic returns a list of Finding records. A finding has an id (<diagnostic>:<subject>, for example crux:strength:r2), the ids of the nodes and relations involved, a headline value, the numbers behind it in details, and a one-line message. Finding.to_dict() gives its JSON form.

Function Reports
missing_parameters An inference variable without a base
unanchored An inference variable with no source and no inferential parent
correlated_support Supports of a node that share a source and no common ancestor, or one variable
claims |logit(stated) − logit(computed)| above a threshold (default 2 ln(11/9) ≈ 0.40)
sensitivity ∂P(T)/∂θ for every parameter θ
crux |∂P(T)/∂θ| · sd(θ): T depends on θ and θ is uncertain
single_points_of_failure Y with P(T | do(Y = false)) below a fraction of P(T) (0.1)
failure_impact Every Y ranked by P(T | do(Y = false)) / P(T), the threshold marking the line
value_of_information The mutual information I(T; Y) in bits

When no parameter T depends on is uncertain, as in a graph of plain numbers, every crux is zero; crux then returns a single finding crux:<target>, with no value, that says so instead of ranking the zeros. Whether T depends on a parameter is decided from the network, not from the size of a derivative, so every engine gives the same answer: θ does not count when it is d-separated from T given the evidence, or when it cancels out of its own table, in the rows the evidence leaves, in exact arithmetic (a requires strength whose premise is observed true, the base of an observed-true child whose supports are all observed, a base multiplied by a requires strength of exactly 1 whose premise is observed false). A derivative that is zero only because contributions through different tables cancel is not detected, and θ is then still ranked. Nor is one that is zero only because θ becomes a common factor once a free parent of its table, neither observed nor T, is summed out: the rows of the table alone do not show it. With T refuting an observed-true X and a free root M requiring X, the requires strength multiplies both values of T alike once M is summed out, yet it is still ranked; observing M as well lets the table show it.

The cancellation check enumerates the rows of the carrier's table that the evidence leaves, in exact rational arithmetic, so its worst-case cost grows like 2^k, where k is the number of the carrier's parents not fixed by evidence, interventions or exclusive constraints; that parent count is the scaling limit. The check returns as soon as it reaches a row that rules cancellation out, so it can stop before the last row, and crux checks only uncertain parameters and stops at the first one T depends on. A parameter that does cancel always costs the full enumeration.

diagnose(graph, target) runs them all. If a parameter is missing the graph cannot be compiled, and the report stops after the structural checks.

Under evidence

Every inference diagnostic, and diagnose, takes evidence, a mapping of node ids to observed values, written E below. Without it, or with {}, each answers for the graph before anything is observed, exactly as before evidence existed. With it:

Diagnostic Under evidence E
claims stated against P(X | E); observed variables are skipped
sensitivity ∂P(T | E)/∂θ, by the quotient rule below
crux |∂P(T | E)/∂θ| · sd(θ), sd(θ) of the parameter's own credence
single_points_of_failure Y with P(T | do(Y = false), E) below a fraction of P(T | E)
failure_impact P(T | do(Y = false), E) / P(T | E), over the same candidates
value_of_information I(T; Y | E) in bits, over the unobserved variables Y

The target may not be observed, directly or through an equivalent node, since its diagnostics would all be trivial; that is a ValidationError. Evidence of probability zero, including equivalent nodes observed at different values, is a ZeroProbabilityError. A single point of failure is not reported for a variable whose failure the evidence rules out, P(E | do(Y = false)) = 0: given what was observed, Y did not fail. Crux keeps sd(θ) from the parameter's credence as written; the parameters are not updated on the evidence, as they are not for the uncertainty bands of a query.

Sensitivity under evidence is not a two-point difference. Write C for the exclusive constraints, observed in every query, and

N(θ) = P(T, E, C) and D(θ) = P(E, C),

so that P(T | E) = N(θ) / D(θ). Each of N and D is a joint probability of the network, so each is a straight line in θ: N(θ) = N(0) + θ N′ with N′ = N(1) − N(0), and likewise D. Their ratio is not a line, so P(T | E, θ = 1) − P(T | E, θ = 0) is not its derivative; the quotient rule is exact instead:

∂P(T | E)/∂θ = (N′ D(θ) − N(θ) D′) / D(θ)²,

from N and D at θ = 0, at θ = 1, and at the value θ holds. The fast two-point identity ∂P(T)/∂θ = P(T | θ = 1) − P(T | θ = 0) may be used only when D does not depend on θ: with no evidence and no exclusive constraint, where D = 1. For one observation o of a hypothesis H with base h, P(o | H) = L and P(o | not H) = b, P(H | o) = hL / (hL + (1 − h) b) runs from 0 at h = 0 to 1 at h = 1, a difference of 1, while its derivative at h = 0.3, L = 0.8, b = 0.2 is Lb / (hL + (1 − h) b)² = 1.108.

Sensitivity is exact. Every joint probability of the network is multilinear in the parameters, so without exclusive constraints or evidence P(T) is a straight line in each θ and ∂P(T)/∂θ = P(T | θ = 1) − P(T | θ = 0). An exclusive constraint C is observed in every query, which makes P(T) = P(T, C) / P(C) a ratio of two such lines; the two-point difference is then not the derivative, and the quotient rule is used instead, as under evidence above.

Claims are compared in log-odds but reported in probability. A claims finding's value is the signed gap stated − computed, and its details hold stated and computed, all probabilities; the threshold it was tested against is in natural log-odds and is named for its unit, threshold_log_odds, so the two scales cannot be confused in the JSON report.

Crux is the primary weak-point ranking. A steep derivative on a parameter known exactly, or a wide credence the target ignores, both score zero.

Single points of failure use intervention, not conditioning: "suppose this premise is simply wrong". The threshold is a fraction of the target's own probability, not a probability: by default a variable is reported when its failure leaves P(T) below a tenth of what it was. An already improbable target therefore still has its premises ranked, rather than every premise whose failure lowers it reported; a threshold of 1 reports all of those.

A premise behind one requires of strength r crosses the threshold only if r > 1 − threshold. Suppose the only directed path from Y to T is one requires relation Y → T of strength r, with no evidence and no exclusive constraint. P(T = 1 | parents) is Y's necessity gate, 1 − r when Y is false and 1 when it is true, times a factor A that does not involve Y. Failing Y leaves T's other parents as they were, so P(T | do(Y = false)) = (1 − r) E[A], while P(T) ≤ E[A] because the gate is at most 1:

P(T | do(Y = false)) / P(T) ≥ 1 − r.

When Y shares no ancestor with T's other parents the ratio is exactly (1 − r) / ((1 − r) + r p), with p = P(Y). At the default threshold of 0.1 such a premise is never a single point of failure when r ≤ 0.9, however probable it is, so an empty list says that no requires is stronger than 0.9, not that nothing is fatal.

Failure impact ranks what lies below the line. failure_impact gives, for every candidate of single_points_of_failure, the ratio P(T | do(Y = false), E) / P(T | E) as its value, lowest first, with P(T | E) as baseline, P(T | do(Y = false), E) as p_target_if_false and the threshold in its details. The threshold does not filter it: single_point_of_failure is true exactly for the variables single_points_of_failure reports. The premises can therefore be ranked by how much their failure dents the target without rerunning at another threshold. When P(T | E) is zero the ratio is undefined and there are no findings.

Parameters are treated as independent, so the most common source of overconfidence in a hand-built argument is a missing common cause: two premises that share an unstated assumption. Model the assumption as a proposition of its own with a relation into each premise, and it will show up in these rankings.

Correlated supports are the same gap in the arguments themselves. The supports relations into a node are a noisy OR of independent reasons, so ten of strength 0.2 into a node of base 0 give it 1 − 0.8¹⁰ = 0.893, whether they are ten experiments or one source's reasoning written down ten times. correlated_support reports, for each node, a group of at least min_supports (2 by default) supporting variables that share a source, anchored in the same document or pointing at the same node through an authored_by or derived_from annotation, when no variable is a common ancestor of them through relations of nonzero strength: an ancestor of every one of them, or one of them that is an ancestor of all the others, as when p0 supports both p1 and the node and p1 supports the node. A variable with no parents and a base of mean exactly 0 or 1 is not a common ancestor: it never varies, so it correlates nothing. Any other common ancestor silences the finding, whatever it stands for and however weak its relations: one broad assumption that every one of them rests on, such as "the instrument works", hides the group even when the supports still come from one source's reasoning. The finding has no value; its details hold supports, the size of the group, and min_supports. The fix is a common parent that each of them requires with strength 1: the group can then lift a node of base b to at most b + (1 − b) c, c the common parent's credence, however large it is. See Reading the diagnostics for a worked example.

Several supports relations from one variable into the same node, such as one from each of two nodes merged as equivalent, are separate terms of the same noisy OR: two of strength 0.2 give a node of base 0 the credence 0.36 where one gives 0.2. correlated_support also reports at least min_supports of them of nonzero strength, with the id correlated-support:<node>:variable:<variable>, whatever the variable's parents: they already share their only input, so no common parent can correlate them. The fix is one relation in their place.

The reference below has an entry for each function and class that credencegraph.diagnostics exports. claims is listed under its own module, credencegraph.diagnostics.claims, whose name it shares. The constants it exports have no entries of their own. The defaults DEFAULT_CLAIM_THRESHOLD, DEFAULT_FAILURE_THRESHOLD and DEFAULT_MIN_SUPPORTS appear by name in the signatures and parameter tables of the functions that take them; the finding kinds, such as OVERCLAIM, and PROVENANCE_TYPES appear in no signature or parameter table. The module credencegraph.diagnostics.weak_points also defines Interventions, interventions, sensitivity_findings, crux_findings, single_point_of_failure_findings and failure_impact_findings, which diagnose and diagnose_many use to build their reports. They are internal to the diagnostics package, not exported by credencegraph.diagnostics and not part of its public API, so they are not documented here; call the diagnostics above instead.

Diagnostics: structural checks, claim consistency, weak points, discharges and sorries.

Structural checks, stated-versus-computed consistency and the weak points of a target each return a list of Finding records, each with an id, the ids of the nodes and relations involved, a headline value and a one-line explanation; Finding.to_dict gives its JSON form. verify instead returns one Check per discharge, holding the findings of its re-check, and sorries returns a SorryReport of the steps a target depends on that rest on judgement only.

Check dataclass

The re-check of one discharge.

Attributes:

Name Type Description
relation str

The id of the relation whose obligation the discharge meets.

kind str

The discharge's kind.

outcome str

passed; flagged when there are findings; unchecked when this package cannot check it, a formal discharge or a citation of a document that is not a local file; or judgement, which has nothing to check.

findings tuple[Finding, ...]

What the re-check found wrong.

status int | None

The exit status of the script's run now, -1 if it timed out; None if no script ran.

reason str | None

Why an unchecked discharge was not checked.

Source code in src/credencegraph/diagnostics/obligations.py
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
@dataclass(frozen=True, slots=True)
class Check:
    """The re-check of one discharge.

    Attributes:
        relation: The id of the relation whose obligation the discharge meets.
        kind: The discharge's kind.
        outcome: ``passed``; ``flagged`` when there are findings; ``unchecked`` when this package
            cannot check it, a ``formal`` discharge or a citation of a document that is not a local
            file; or ``judgement``, which has nothing to check.
        findings: What the re-check found wrong.
        status: The exit status of the script's run now, ``-1`` if it timed out; ``None`` if no
            script ran.
        reason: Why an unchecked discharge was not checked.
    """

    relation: str
    kind: str
    outcome: str
    findings: tuple[Finding, ...] = ()
    status: int | None = None
    reason: str | None = None

    def to_dict(self) -> dict[str, JSON]:
        """Return the check as JSON-ready data.

        Returns:
            ``{"relation", "kind", "outcome", "status", "reason", "findings"}``.
        """
        return {
            "relation": self.relation,
            "kind": self.kind,
            "outcome": self.outcome,
            "status": self.status,
            "reason": self.reason,
            "findings": [finding.to_dict() for finding in self.findings],
        }

to_dict()

Return the check as JSON-ready data.

Returns:

Type Description
dict[str, JSON]

{"relation", "kind", "outcome", "status", "reason", "findings"}.

Source code in src/credencegraph/diagnostics/obligations.py
210
211
212
213
214
215
216
217
218
219
220
221
222
223
def to_dict(self) -> dict[str, JSON]:
    """Return the check as JSON-ready data.

    Returns:
        ``{"relation", "kind", "outcome", "status", "reason", "findings"}``.
    """
    return {
        "relation": self.relation,
        "kind": self.kind,
        "outcome": self.outcome,
        "status": self.status,
        "reason": self.reason,
        "findings": [finding.to_dict() for finding in self.findings],
    }

Finding dataclass

One result of a diagnostic.

Attributes:

Name Type Description
id str

Identifies the finding within a report: <diagnostic>:<subject>, where the subject is a node id, base:<node id> or strength:<relation id> for a parameter, or <node id>:<provenance> for a group of supports, such as claim:document:doi:10.1/x.

diagnostic str

The diagnostic that produced it, such as "crux".

message str

A one-line explanation for a human reader.

nodes tuple[str, ...]

The ids of the nodes involved.

relations tuple[str, ...]

The ids of the relations involved.

value float | None

The diagnostic's headline number, or None for a purely structural finding.

details Mapping[str, float]

The other numbers behind the finding, by name.

Source code in src/credencegraph/diagnostics/records.py
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
@dataclass(frozen=True, slots=True)
class Finding:
    """One result of a diagnostic.

    Attributes:
        id: Identifies the finding within a report: ``<diagnostic>:<subject>``, where the subject is
            a node id, ``base:<node id>`` or ``strength:<relation id>`` for a parameter, or
            ``<node id>:<provenance>`` for a group of supports, such as ``claim:document:doi:10.1/x``.
        diagnostic: The diagnostic that produced it, such as ``"crux"``.
        message: A one-line explanation for a human reader.
        nodes: The ids of the nodes involved.
        relations: The ids of the relations involved.
        value: The diagnostic's headline number, or ``None`` for a purely structural finding.
        details: The other numbers behind the finding, by name.
    """

    id: str
    diagnostic: str
    message: str
    nodes: tuple[str, ...] = ()
    relations: tuple[str, ...] = ()
    value: float | None = None
    details: Mapping[str, float] = field(default_factory=dict)

    def __post_init__(self) -> None:
        """Store read-only copies of the id tuples and the details."""
        object.__setattr__(self, "nodes", tuple(self.nodes))
        object.__setattr__(self, "relations", tuple(self.relations))
        object.__setattr__(self, "details", MappingProxyType(dict(self.details)))

    def to_dict(self) -> dict[str, JSON]:
        """Return the finding as JSON-ready data.

        Returns:
            ``{"id", "diagnostic", "nodes", "relations", "value", "details", "message"}``.
        """
        return {
            "id": self.id,
            "diagnostic": self.diagnostic,
            "nodes": list(self.nodes),
            "relations": list(self.relations),
            "value": self.value,
            "details": dict(self.details),
            "message": self.message,
        }

__post_init__()

Store read-only copies of the id tuples and the details.

Source code in src/credencegraph/diagnostics/records.py
54
55
56
57
58
def __post_init__(self) -> None:
    """Store read-only copies of the id tuples and the details."""
    object.__setattr__(self, "nodes", tuple(self.nodes))
    object.__setattr__(self, "relations", tuple(self.relations))
    object.__setattr__(self, "details", MappingProxyType(dict(self.details)))

to_dict()

Return the finding as JSON-ready data.

Returns:

Type Description
dict[str, JSON]

{"id", "diagnostic", "nodes", "relations", "value", "details", "message"}.

Source code in src/credencegraph/diagnostics/records.py
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
def to_dict(self) -> dict[str, JSON]:
    """Return the finding as JSON-ready data.

    Returns:
        ``{"id", "diagnostic", "nodes", "relations", "value", "details", "message"}``.
    """
    return {
        "id": self.id,
        "diagnostic": self.diagnostic,
        "nodes": list(self.nodes),
        "relations": list(self.relations),
        "value": self.value,
        "details": dict(self.details),
        "message": self.message,
    }

Report dataclass

The findings for a graph and for each of several target nodes.

Attributes:

Name Type Description
findings tuple[Finding, ...]

The graph-wide findings: missing parameters, unanchored variables, correlated supports, overclaims and underclaims. They do not depend on a target and appear once.

weak_points Mapping[str, tuple[Finding, ...]]

Each target's sensitivity, crux, single points of failure, failure impact and value of information, by target id in the order the targets were given. A finding id is unique within one target's list; the same id, such as crux:strength:r1, can recur for another target.

Source code in src/credencegraph/diagnostics/report.py
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
@dataclass(frozen=True, slots=True)
class Report:
    """The findings for a graph and for each of several target nodes.

    Attributes:
        findings: The graph-wide findings: missing parameters, unanchored variables, correlated
            supports, overclaims and underclaims. They do not depend on a target and appear once.
        weak_points: Each target's sensitivity, crux, single points of failure, failure impact and
            value of information, by target id in the order the targets were given. A finding id is
            unique within one target's list; the same id, such as ``crux:strength:r1``, can recur for
            another target.
    """

    findings: tuple[Finding, ...] = ()
    weak_points: Mapping[str, tuple[Finding, ...]] = field(default_factory=dict)

    def __post_init__(self) -> None:
        """Store read-only copies of the findings and of the weak points."""
        object.__setattr__(self, "findings", tuple(self.findings))
        object.__setattr__(
            self,
            "weak_points",
            MappingProxyType({target: tuple(found) for target, found in self.weak_points.items()}),
        )

    def all_findings(self) -> list[Finding]:
        """Return the graph-wide findings, then each target's.

        Returns:
            Every finding, in report order.
        """
        return [*self.findings, *(finding for found in self.weak_points.values() for finding in found)]

    def to_dicts(self) -> list[dict[str, JSON]]:
        """Return every finding as JSON-ready data, naming the target of each.

        Returns:
            The graph-wide findings, then each target's, as ``Finding.to_dict`` with a ``"target"``
            key: ``None`` for a graph-wide finding, and the target's id for a weak point.
        """
        records = [{**finding.to_dict(), "target": None} for finding in self.findings]
        for target, found in self.weak_points.items():
            records.extend({**finding.to_dict(), "target": target} for finding in found)
        return records

__post_init__()

Store read-only copies of the findings and of the weak points.

Source code in src/credencegraph/diagnostics/report.py
46
47
48
49
50
51
52
53
def __post_init__(self) -> None:
    """Store read-only copies of the findings and of the weak points."""
    object.__setattr__(self, "findings", tuple(self.findings))
    object.__setattr__(
        self,
        "weak_points",
        MappingProxyType({target: tuple(found) for target, found in self.weak_points.items()}),
    )

all_findings()

Return the graph-wide findings, then each target's.

Returns:

Type Description
list[Finding]

Every finding, in report order.

Source code in src/credencegraph/diagnostics/report.py
55
56
57
58
59
60
61
def all_findings(self) -> list[Finding]:
    """Return the graph-wide findings, then each target's.

    Returns:
        Every finding, in report order.
    """
    return [*self.findings, *(finding for found in self.weak_points.values() for finding in found)]

to_dicts()

Return every finding as JSON-ready data, naming the target of each.

Returns:

Name Type Description
list[dict[str, JSON]]

The graph-wide findings, then each target's, as Finding.to_dict with a "target"

key list[dict[str, JSON]]

None for a graph-wide finding, and the target's id for a weak point.

Source code in src/credencegraph/diagnostics/report.py
63
64
65
66
67
68
69
70
71
72
73
def to_dicts(self) -> list[dict[str, JSON]]:
    """Return every finding as JSON-ready data, naming the target of each.

    Returns:
        The graph-wide findings, then each target's, as ``Finding.to_dict`` with a ``"target"``
        key: ``None`` for a graph-wide finding, and the target's id for a weak point.
    """
    records = [{**finding.to_dict(), "target": None} for finding in self.findings]
    for target, found in self.weak_points.items():
        records.extend({**finding.to_dict(), "target": target} for finding in found)
    return records

Sorry dataclass

A step a target depends on that rests on judgement only.

Attributes:

Name Type Description
relation str

The relation's id.

type str

The relation's type.

source str

Its source node.

target str

Its target node.

reason str

Why it counts as a sorry, one of SORRY_REASONS.

obligation str | None

The obligation's statement, if the relation has one.

sensitivity float | None

dP(target)/d strength, or None for a relation without a strength.

Source code in src/credencegraph/diagnostics/obligations.py
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
@dataclass(frozen=True, slots=True)
class Sorry:
    """A step a target depends on that rests on judgement only.

    Attributes:
        relation: The relation's id.
        type: The relation's type.
        source: Its source node.
        target: Its target node.
        reason: Why it counts as a sorry, one of ``SORRY_REASONS``.
        obligation: The obligation's statement, if the relation has one.
        sensitivity: ``dP(target)/d strength``, or ``None`` for a relation without a strength.
    """

    relation: str
    type: str
    source: str
    target: str
    reason: str
    obligation: str | None = None
    sensitivity: float | None = None

    def to_dict(self) -> dict[str, JSON]:
        """Return the sorry as JSON-ready data.

        Returns:
            ``{"relation", "type", "source", "target", "reason", "obligation", "sensitivity"}``.
        """
        return {
            "relation": self.relation,
            "type": self.type,
            "source": self.source,
            "target": self.target,
            "reason": self.reason,
            "obligation": self.obligation,
            "sensitivity": self.sensitivity,
        }

to_dict()

Return the sorry as JSON-ready data.

Returns:

Type Description
dict[str, JSON]

{"relation", "type", "source", "target", "reason", "obligation", "sensitivity"}.

Source code in src/credencegraph/diagnostics/obligations.py
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
def to_dict(self) -> dict[str, JSON]:
    """Return the sorry as JSON-ready data.

    Returns:
        ``{"relation", "type", "source", "target", "reason", "obligation", "sensitivity"}``.
    """
    return {
        "relation": self.relation,
        "type": self.type,
        "source": self.source,
        "target": self.target,
        "reason": self.reason,
        "obligation": self.obligation,
        "sensitivity": self.sensitivity,
    }

SorryReport dataclass

The steps a target depends on, split into the discharged ones and the sorries.

Attributes:

Name Type Description
target str

The target node.

steps tuple[str, ...]

The ids of the inferential relations the target depends on, in the graph's order.

discharged tuple[str, ...]

The ids of those relations whose obligation is discharged by more than judgement.

sorries tuple[Sorry, ...]

The rest, by decreasing absolute sensitivity, those without a strength last.

Source code in src/credencegraph/diagnostics/obligations.py
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
@dataclass(frozen=True, slots=True)
class SorryReport:
    """The steps a target depends on, split into the discharged ones and the sorries.

    Attributes:
        target: The target node.
        steps: The ids of the inferential relations the target depends on, in the graph's order.
        discharged: The ids of those relations whose obligation is discharged by more than judgement.
        sorries: The rest, by decreasing absolute sensitivity, those without a strength last.
    """

    target: str
    steps: tuple[str, ...]
    discharged: tuple[str, ...]
    sorries: tuple[Sorry, ...] = ()

    def to_dict(self) -> dict[str, JSON]:
        """Return the report as JSON-ready data.

        Returns:
            ``{"target", "steps", "discharged", "sorries"}``.
        """
        return {
            "target": self.target,
            "steps": list(self.steps),
            "discharged": list(self.discharged),
            "sorries": [sorry.to_dict() for sorry in self.sorries],
        }

to_dict()

Return the report as JSON-ready data.

Returns:

Type Description
dict[str, JSON]

{"target", "steps", "discharged", "sorries"}.

Source code in src/credencegraph/diagnostics/obligations.py
442
443
444
445
446
447
448
449
450
451
452
453
def to_dict(self) -> dict[str, JSON]:
    """Return the report as JSON-ready data.

    Returns:
        ``{"target", "steps", "discharged", "sorries"}``.
    """
    return {
        "target": self.target,
        "steps": list(self.steps),
        "discharged": list(self.discharged),
        "sorries": [sorry.to_dict() for sorry in self.sorries],
    }

can_fail(source)

Tell whether a Python script has a reachable way to fail: an assert, a raise, or a non-zero exit.

Code that a constant condition makes unreachable is ignored first: the body of if False:, if 0: or while False:, the else of if True:, the dead branch of a conditional expression, and an assert of a constantly true condition such as assert True. A condition counts as constant when it is a literal, the not of a constant condition, or a conditional expression whose condition and taken branch are constant, at any depth of nesting, so if (False if True else False): is pruned like if False: and assert (True if True else False) cannot fail; a check behind any other condition is taken to be reachable. A script that does not parse has no checked failure path, only a crash, and counts as unable to fail.

Parameters:

Name Type Description Default
source str

The script's text.

required

Returns:

Type Description
bool

True when the reachable code holds an assert or raise statement, or an exit call

bool

(sys.exit, os._exit, exit or quit) whose argument is not 0 or None.

Source code in src/credencegraph/diagnostics/obligations.py
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
def can_fail(source: str) -> bool:
    """Tell whether a Python script has a reachable way to fail: an ``assert``, a ``raise``, or a non-zero exit.

    Code that a constant condition makes unreachable is ignored first: the body of ``if False:``,
    ``if 0:`` or ``while False:``, the ``else`` of ``if True:``, the dead branch of a conditional
    expression, and an ``assert`` of a constantly true condition such as ``assert True``. A condition
    counts as constant when it is a literal, the ``not`` of a constant condition, or a conditional
    expression whose condition and taken branch are constant, at any depth of nesting, so
    ``if (False if True else False):`` is pruned like ``if False:`` and
    ``assert (True if True else False)`` cannot fail; a check behind any other condition is taken to be
    reachable. A script that does not parse has no checked failure path, only a crash, and counts as
    unable to fail.

    Args:
        source: The script's text.

    Returns:
        ``True`` when the reachable code holds an ``assert`` or ``raise`` statement, or an exit call
        (``sys.exit``, ``os._exit``, ``exit`` or ``quit``) whose argument is not ``0`` or ``None``.
    """
    try:
        tree = _DeadCode().visit(ast.parse(source))
    except (SyntaxError, ValueError):
        return False
    return any(
        isinstance(node, ast.Assert | ast.Raise) or (isinstance(node, ast.Call) and _exits_nonzero(node))
        for node in ast.walk(tree)
    )

correlated_support(graph, min_supports=DEFAULT_MIN_SUPPORTS)

Find supports of one node that share a source but no common ancestor.

The supports relations into a node combine as a noisy OR of independent reasons: ten of strength 0.2 into a node of base 0 give it 1 - 0.8^10, about 0.893. Reasons that come from one source are rarely independent, and nothing in the graph says so unless they share an ancestor. Two supporting variables share a source when a node of each is anchored in the same document, or points at the same node through an authored_by or derived_from annotation. A group of at least min_supports supporting variables of one node that share a source is reported when no variable is a common ancestor of them: an ancestor, through chains of requires, supports or refutes relations, of every one of them, or one of them that is an ancestor of all the others, as in p0 supports p1, p0 supports claim, p1 supports claim. Either way the graph already makes them depend on one variable, so inference does not count them as independent. A relation of strength exactly 0 does not make a parent, nor a support: it leaves the child's probability the same whatever the parent's value. Nor is a fixed point a common ancestor, nor an ancestor through it: a variable that always takes the same value, whether it has no parents and a base of mean exactly 0 or 1 or has a table that gives it that value whatever its parents are, as with a base of 1 and only supports parents, correlates nothing and passes nothing on. The usual fix is a common parent: a proposition such as "this source's unverified reasoning is sound", which each of them requires with strength 1, so that they fail together. A node of base b with no other parents then gets at most b + (1 - b) c from the group, c the credence of the proposition, however large the group.

Nodes merged by equivalent relations are one variable, named after the earliest of them, with the sources of all of them. A group that shares several sources is reported once, under the first.

Several supports relations from one variable into the same node, for example one from each of two merged nodes, are separate terms of the noisy OR too: two of strength 0.2 lift a node of base 0 to 0.36 where one gives 0.2. At least min_supports of them of nonzero strength are reported whatever the variable's parents, since no common parent can correlate terms that already share their only input; the fix is one relation in their place.

Parameters:

Name Type Description Default
graph Graph

The graph.

required
min_supports int

The smallest group reported, at least 2; the default, 2, reports any two.

DEFAULT_MIN_SUPPORTS

Returns:

Type Description
list[Finding]

One finding per node and group, the nodes in insertion order, and for each node its groups

list[Finding]

before its repeated supports. Its id is correlated-support:<node>:<kind>:<source>, where

list[Finding]

the kind is document or the annotation type, or variable with the supporting variable

list[Finding]

as the source for supports repeated from one variable. Its nodes are the supported node, then

list[Finding]

the group or the variable; its relations are their supports relations of nonzero strength

list[Finding]

into the node. It has no value; its details are

list[Finding]

supports, the size of the group or the number of relations, and min_supports.

Raises:

Type Description
ValidationError

If min_supports is not an integer >= 2.

Source code in src/credencegraph/diagnostics/structure.py
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
def correlated_support(graph: Graph, min_supports: int = DEFAULT_MIN_SUPPORTS) -> list[Finding]:
    """Find supports of one node that share a source but no common ancestor.

    The ``supports`` relations into a node combine as a noisy OR of independent reasons: ten of
    strength 0.2 into a node of base 0 give it 1 - 0.8^10, about 0.893. Reasons that come from one
    source are rarely independent, and nothing in the graph says so unless they share an ancestor. Two
    supporting variables share a source when a node of each is anchored in the same document, or
    points at the same node through an ``authored_by`` or ``derived_from`` annotation. A group of at
    least ``min_supports`` supporting variables of one node that share a source is reported when no
    variable is a common ancestor of them: an ancestor, through chains of ``requires``, ``supports``
    or ``refutes`` relations, of every one of them, or one of them that is an ancestor of all the
    others, as in ``p0 supports p1``, ``p0 supports claim``, ``p1 supports claim``. Either way the
    graph already makes them depend on one variable, so inference does not count them as independent.
    A relation of strength exactly 0 does not make a parent, nor a support: it leaves the child's
    probability the same whatever the parent's value. Nor is a fixed point a common ancestor, nor an
    ancestor through it: a variable that always takes the same value, whether it has no parents and
    a base of mean exactly 0 or 1 or has a table that gives it that value whatever its parents are,
    as with a base of 1 and only ``supports`` parents, correlates nothing and passes nothing on.
    The usual fix is a common parent: a proposition such as "this source's unverified reasoning
    is sound", which each of them requires with strength 1, so that they fail together. A node of
    base b with no other parents then gets at most b + (1 - b) c from the group, c the credence of
    the proposition, however large the group.

    Nodes merged by ``equivalent`` relations are one variable, named after the earliest of them, with
    the sources of all of them. A group that shares several sources is reported once, under the first.

    Several ``supports`` relations from one variable into the same node, for example one from each
    of two merged nodes, are separate terms of the noisy OR too: two of strength 0.2 lift a node of
    base 0 to 0.36 where one gives 0.2. At least ``min_supports`` of them of nonzero strength are
    reported whatever the variable's parents, since no common parent can correlate terms that
    already share their only input; the fix is one relation in their place.

    Args:
        graph: The graph.
        min_supports: The smallest group reported, at least 2; the default, 2, reports any two.

    Returns:
        One finding per node and group, the nodes in insertion order, and for each node its groups
        before its repeated supports. Its id is ``correlated-support:<node>:<kind>:<source>``, where
        the kind is ``document`` or the annotation type, or ``variable`` with the supporting variable
        as the source for supports repeated from one variable. Its nodes are the supported node, then
        the group or the variable; its relations are their ``supports`` relations of nonzero strength
        into the node. It has no value; its details are
        ``supports``, the size of the group or the number of relations, and ``min_supports``.

    Raises:
        ValidationError: If ``min_supports`` is not an integer >= 2.
    """
    count = _min_supports(min_supports)
    members = inference_sets(graph)
    root = {node_id: representative for representative, ids in members.items() for node_id in ids}
    provenance = _provenance(graph, members, root)
    parents: dict[str, set[str]] = {}
    lineage: dict[str, set[str]] = {}
    supports: dict[str, dict[str, list[str]]] = {}
    terms: dict[str, list[tuple[str, str, Fraction]]] = {}
    for relation in graph.relations.values():
        if relation.type in STRENGTH_TYPES:
            parent, child = root[relation.source], root[relation.target]
            # A term of strength 0 multiplies the child's table by 1 - 0 = 1 at both of the parent's
            # values, so it makes the parent no cause of the child; a Beta strength has a positive mean.
            if relation.strength is not None and relation.strength.mean > 0.0:
                parents.setdefault(child, set()).add(parent)
                terms.setdefault(child, []).append((relation.type, parent, Fraction(relation.strength.mean)))
                if relation.type == SUPPORTS:
                    supports.setdefault(child, {}).setdefault(parent, []).append(relation.id)
    fixed = _fixed_points(graph, members, parents, terms)
    # A fixed variable takes its value whatever its parents are, so nothing above it reaches its descendants.
    effective = {child: own for child, own in parents.items() if child not in fixed}
    findings: list[Finding] = []
    for target in members:
        by_parent = supports.get(target, {})
        sharing: dict[tuple[str, str], list[str]] = {}
        for parent in by_parent:
            for key in provenance[parent]:
                sharing.setdefault(key, []).append(parent)
        groups: dict[tuple[str, ...], list[tuple[str, str]]] = {}
        for key, group in sharing.items():
            common = set.intersection(*(_lineage(parent, effective, lineage) for parent in group))
            if len(group) >= count and not common - fixed.keys():
                groups.setdefault(tuple(group), []).append(key)
        for group, keys in groups.items():
            names = ", ".join(repr(parent) for parent in group)
            findings.append(
                Finding(
                    id=f"{CORRELATED_SUPPORT}:{target}:{keys[0][0]}:{keys[0][1]}",
                    diagnostic=CORRELATED_SUPPORT,
                    message=(
                        f"node {target!r} has {len(group)} supports, {names}, that share "
                        f"{' and '.join(_render(key) for key in keys)} but no common ancestor: inference counts "
                        "them as independent reasons; if they rest on one source's reasoning, add a proposition "
                        "for it that each of them requires"
                    ),
                    nodes=(target, *group),
                    relations=tuple(relation for parent in group for relation in by_parent[parent]),
                    details={"supports": float(len(group)), "min_supports": float(count)},
                )
            )
        findings.extend(
            Finding(
                id=f"{CORRELATED_SUPPORT}:{target}:variable:{parent}",
                diagnostic=CORRELATED_SUPPORT,
                message=(
                    f"node {target!r} has {len(relations)} supports from one variable, "
                    f"{_describe(members[parent])}: inference counts them as independent reasons; "
                    "if they are one reason, replace them with a single supports relation"
                ),
                nodes=(target, parent),
                relations=tuple(relations),
                details={"supports": float(len(relations)), "min_supports": float(count)},
            )
            for parent, relations in by_parent.items()
            if len(relations) >= count
        )
    return findings

crux(network, target, *, evidence=None, engine=None)

Rank the parameters by |dP(target | evidence)/dθ| * sd(θ).

A parameter is a crux when the target depends on it and it is uncertain: a steep derivative on a Point parameter, or a wide credence the target ignores, both score zero. This is the primary weak-point ranking. Under evidence, sd(θ) is still that of the parameter's own credence: the parameters are not updated on the evidence, as they are not for the uncertainty bands.

Parameters:

Name Type Description Default
network Network

The network.

required
target str

The id of the target node.

required
evidence Mapping[str, bool] | None

Node ids and their observed values; none by default.

None
engine Engine | None

The exact engine; variable elimination by default.

None

Returns:

Type Description
list[Finding]

One finding per parameter, largest crux first; or a single structural finding, with no

list[Finding]

value, when no parameter the target depends on is uncertain, as when every one is a

list[Finding]

Point.

Raises:

Type Description
ValidationError

If target or an evidence node is not an inference variable of the network, or the evidence observes the target.

ZeroProbabilityError

If the evidence and the exclusive constraints have probability zero.

Source code in src/credencegraph/diagnostics/weak_points.py
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
def crux(
    network: Network,
    target: str,
    *,
    evidence: Mapping[str, bool] | None = None,
    engine: Engine | None = None,
) -> list[Finding]:
    """Rank the parameters by ``|dP(target | evidence)/dθ| * sd(θ)``.

    A parameter is a crux when the target depends on it *and* it is uncertain: a steep derivative on
    a ``Point`` parameter, or a wide credence the target ignores, both score zero. This is the
    primary weak-point ranking. Under evidence, sd(θ) is still that of the parameter's own credence:
    the parameters are not updated on the evidence, as they are not for the uncertainty bands.

    Args:
        network: The network.
        target: The id of the target node.
        evidence: Node ids and their observed values; none by default.
        engine: The exact engine; variable elimination by default.

    Returns:
        One finding per parameter, largest crux first; or a single structural finding, with no
        value, when no parameter the target depends on is uncertain, as when every one is a
        ``Point``.

    Raises:
        ValidationError: If ``target`` or an evidence node is not an inference variable of the
            network, or the evidence observes the target.
        ZeroProbabilityError: If the evidence and the ``exclusive`` constraints have probability zero.
    """
    slopes = derivatives(network, target, evidence=evidence, engine=engine)
    return crux_findings(network, target, slopes, evidence=evidence)

dependencies(graph, target)

Find the inferential relations a target depends on.

These are the requires, supports and refutes relations into the target or into any node it depends on, and the equivalent and exclusive relations that touch either; the other end of each also becomes a node the target depends on.

Parameters:

Name Type Description Default
graph Graph

The graph.

required
target str

The target node's id.

required

Returns:

Type Description
list[Relation]

The relations, in the graph's order.

Source code in src/credencegraph/diagnostics/obligations.py
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
def dependencies(graph: Graph, target: str) -> list[Relation]:
    """Find the inferential relations a target depends on.

    These are the ``requires``, ``supports`` and ``refutes`` relations into the target or into any
    node it depends on, and the ``equivalent`` and ``exclusive`` relations that touch either; the
    other end of each also becomes a node the target depends on.

    Args:
        graph: The graph.
        target: The target node's id.

    Returns:
        The relations, in the graph's order.
    """
    reached = {target}
    found: set[str] = set()
    grew = True
    while grew:
        grew = False
        for relation in graph.relations.values():
            if relation.id in found:
                continue
            if relation.type in STRENGTH_TYPES:
                touches = relation.target in reached
            elif relation.type in {EQUIVALENT, EXCLUSIVE}:
                touches = relation.source in reached or relation.target in reached
            else:
                continue
            if touches:
                found.add(relation.id)
                reached.update((relation.source, relation.target))
                grew = True
    return [relation for relation in graph.relations.values() if relation.id in found]

derivatives(network, target, *, evidence=None, engine=None)

Compute dP(target | evidence)/dθ exactly for every parameter θ of the network.

See the module documentation for why this is exact. Each derivative is taken at the value the parameter currently holds, with every other parameter held at its own. With evidence the derivative is the quotient rule on two multilinear evaluations, not the two-point difference.

Parameters:

Name Type Description Default
network Network

The network.

required
target str

The id of the target node.

required
evidence Mapping[str, bool] | None

Node ids and their observed values; none by default.

None
engine Engine | None

The exact engine; variable elimination by default.

None

Returns:

Type Description
dict[ParameterKey, float]

The derivative of P(target = true | evidence) by parameter, in the order

dict[ParameterKey, float]

network.parameters lists them.

Raises:

Type Description
ValidationError

If target or an evidence node is not an inference variable of the network, or the evidence observes the target.

ZeroProbabilityError

If the evidence and the exclusive constraints have probability zero.

Source code in src/credencegraph/diagnostics/weak_points.py
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
def derivatives(
    network: Network,
    target: str,
    *,
    evidence: Mapping[str, bool] | None = None,
    engine: Engine | None = None,
) -> dict[ParameterKey, float]:
    """Compute ``dP(target | evidence)/dθ`` exactly for every parameter θ of the network.

    See the module documentation for why this is exact. Each derivative is taken at the value the
    parameter currently holds, with every other parameter held at its own. With evidence the
    derivative is the quotient rule on two multilinear evaluations, not the two-point difference.

    Args:
        network: The network.
        target: The id of the target node.
        evidence: Node ids and their observed values; none by default.
        engine: The exact engine; variable elimination by default.

    Returns:
        The derivative of ``P(target = true | evidence)`` by parameter, in the order
        ``network.parameters`` lists them.

    Raises:
        ValidationError: If ``target`` or an evidence node is not an inference variable of the
            network, or the evidence observes the target.
        ZeroProbabilityError: If the evidence and the ``exclusive`` constraints have probability zero.
    """
    engine = engine or VariableElimination()
    query = _observed(network, target, evidence)
    index = network.index(target)
    conditions = _conditions(network, query.evidence)
    numerator, denominator = _parts(network, index, conditions, engine)
    if denominator <= 0.0:
        msg = "the evidence, together with the exclusive constraints, has probability zero"
        raise ZeroProbabilityError(msg)
    slopes: dict[ParameterKey, float] = {}
    for key in network.parameters:
        low_n, low_d = _parts(network.with_parameters({key: 0.0}), index, conditions, engine)
        high_n, high_d = _parts(network.with_parameters({key: 1.0}), index, conditions, engine)
        slopes[key] = ((high_n - low_n) * denominator - numerator * (high_d - low_d)) / denominator**2
    return slopes

diagnose(graph, target=None, *, evidence=None, claim_threshold=DEFAULT_CLAIM_THRESHOLD, failure_threshold=DEFAULT_FAILURE_THRESHOLD, min_supports=DEFAULT_MIN_SUPPORTS, engine=None)

Run the diagnostics that apply to a graph and, optionally, a target node.

The structural checks (missing parameters, unanchored variables, correlated supports) always run. If a parameter is missing the graph cannot be compiled, and the report stops there. Otherwise the stated credences are compared with the computed ones, and, when a target is given, its sensitivity, crux, single points of failure, failure impact and value of information follow. diagnose_many does the same for several targets, running the graph-wide checks once.

With evidence every inference diagnostic is taken given it: the computed credences are P(X | evidence), the sensitivities and cruxes are derivatives of P(target | evidence), the single points of failure and the failure impact compare P(target | do(Y = false), evidence) with P(target | evidence), and the value of information is I(target; Y | evidence). Observed variables are left out of the comparisons and rankings over nodes, and the target may not be observed. Without evidence the report is the one for the graph before anything is observed.

Parameters:

Name Type Description Default
graph Graph

The graph.

required
target str | None

The id of the node whose weak points are wanted, if any.

None
evidence Mapping[str, bool] | None

Node ids and their observed values; none by default.

None
claim_threshold float

The threshold passed to claims, a gap in natural log-odds; the default, 2 ln(11 / 9), is a convention of this package.

DEFAULT_CLAIM_THRESHOLD
failure_threshold float

The threshold passed to single_points_of_failure, and marked in the failure_impact ranking, a fraction of the target's own probability; the default, 0.1, is a convention of this package. A premise behind one requires of strength r crosses it only if r > 1 - threshold.

DEFAULT_FAILURE_THRESHOLD
min_supports int

The smallest group passed to correlated_support, at least 2.

DEFAULT_MIN_SUPPORTS
engine Engine | None

The exact engine; variable elimination by default.

None

Returns:

Type Description
list[Finding]

The findings, grouped by diagnostic in the order above; [f.to_dict() for f in ...] is

list[Finding]

ready for json.dumps.

Raises:

Type Description
ValidationError

If a threshold or min_supports is out of range, target or an evidence node is not an inference variable, or the evidence observes the target.

CompileError

If the graph has no missing parameter but still fails to compile, for example over conflicting bases on merged nodes.

ZeroProbabilityError

If the evidence and the exclusive constraints have probability zero.

Source code in src/credencegraph/diagnostics/report.py
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
def diagnose(  # noqa: PLR0913 - the options after the target are keyword-only
    graph: Graph,
    target: str | None = None,
    *,
    evidence: Mapping[str, bool] | None = None,
    claim_threshold: float = DEFAULT_CLAIM_THRESHOLD,
    failure_threshold: float = DEFAULT_FAILURE_THRESHOLD,
    min_supports: int = DEFAULT_MIN_SUPPORTS,
    engine: Engine | None = None,
) -> list[Finding]:
    """Run the diagnostics that apply to a graph and, optionally, a target node.

    The structural checks (missing parameters, unanchored variables, correlated supports) always
    run. If a parameter is missing the graph cannot be compiled, and the report stops there. Otherwise the stated credences
    are compared with the computed ones, and, when a target is given, its sensitivity, crux,
    single points of failure, failure impact and value of information follow. ``diagnose_many``
    does the same for several targets, running the graph-wide checks once.

    With evidence every inference diagnostic is taken given it: the computed credences are
    ``P(X | evidence)``, the sensitivities and cruxes are derivatives of ``P(target | evidence)``,
    the single points of failure and the failure impact compare ``P(target | do(Y = false), evidence)``
    with ``P(target | evidence)``, and the value of information is ``I(target; Y | evidence)``. Observed
    variables are left out of the comparisons and rankings over nodes, and the target may not be
    observed. Without evidence the report is the one for the graph before anything is observed.

    Args:
        graph: The graph.
        target: The id of the node whose weak points are wanted, if any.
        evidence: Node ids and their observed values; none by default.
        claim_threshold: The threshold passed to ``claims``, a gap in natural log-odds; the default,
            ``2 ln(11 / 9)``, is a convention of this package.
        failure_threshold: The threshold passed to ``single_points_of_failure``, and marked in the
            ``failure_impact`` ranking, a fraction of the target's own probability; the default, 0.1,
            is a convention of this package. A premise behind one ``requires`` of strength r crosses
            it only if r > 1 - threshold.
        min_supports: The smallest group passed to ``correlated_support``, at least 2.
        engine: The exact engine; variable elimination by default.

    Returns:
        The findings, grouped by diagnostic in the order above; ``[f.to_dict() for f in ...]`` is
        ready for ``json.dumps``.

    Raises:
        ValidationError: If a threshold or ``min_supports`` is out of range, ``target`` or an evidence
            node is not an inference variable, or the evidence observes the target.
        CompileError: If the graph has no missing parameter but still fails to compile, for example
            over conflicting bases on merged nodes.
        ZeroProbabilityError: If the evidence and the ``exclusive`` constraints have probability zero.
    """
    report = diagnose_many(
        graph,
        () if target is None else (target,),
        evidence=evidence,
        claim_threshold=claim_threshold,
        failure_threshold=failure_threshold,
        min_supports=min_supports,
        engine=engine,
    )
    return report.all_findings()

diagnose_many(graph, targets, *, evidence=None, claim_threshold=DEFAULT_CLAIM_THRESHOLD, failure_threshold=DEFAULT_FAILURE_THRESHOLD, min_supports=DEFAULT_MIN_SUPPORTS, engine=None)

Run the graph-wide diagnostics once and the weak-point diagnostics for each of several targets.

The graph-wide findings are those diagnose reports without a target; each target's weak points are those diagnose adds for it, under the same evidence and thresholds. A target named twice is diagnosed once. If a parameter is missing the graph cannot be compiled, and every target's list is empty.

Parameters:

Name Type Description Default
graph Graph

The graph.

required
targets Iterable[str]

The ids of the nodes whose weak points are wanted; may be empty.

required
evidence Mapping[str, bool] | None

Node ids and their observed values; none by default.

None
claim_threshold float

The threshold passed to claims, a gap in natural log-odds.

DEFAULT_CLAIM_THRESHOLD
failure_threshold float

The threshold passed to single_points_of_failure, a fraction of each target's own probability.

DEFAULT_FAILURE_THRESHOLD
min_supports int

The smallest group passed to correlated_support, at least 2.

DEFAULT_MIN_SUPPORTS
engine Engine | None

The exact engine; variable elimination by default.

None

Returns:

Type Description
Report

The graph-wide findings, and each target's weak points in the order the targets were given;

Report

report.to_dicts() is ready for json.dumps.

Raises:

Type Description
ValidationError

If targets is a single string, a threshold or min_supports is out of range, a target or an evidence node is not an inference variable, or the evidence observes a target.

CompileError

If the graph has no missing parameter but still fails to compile, for example over conflicting bases on merged nodes.

ZeroProbabilityError

If the evidence and the exclusive constraints have probability zero.

Source code in src/credencegraph/diagnostics/report.py
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
def diagnose_many(  # noqa: PLR0913 - the options after the targets are keyword-only
    graph: Graph,
    targets: Iterable[str],
    *,
    evidence: Mapping[str, bool] | None = None,
    claim_threshold: float = DEFAULT_CLAIM_THRESHOLD,
    failure_threshold: float = DEFAULT_FAILURE_THRESHOLD,
    min_supports: int = DEFAULT_MIN_SUPPORTS,
    engine: Engine | None = None,
) -> Report:
    """Run the graph-wide diagnostics once and the weak-point diagnostics for each of several targets.

    The graph-wide findings are those ``diagnose`` reports without a target; each target's weak
    points are those ``diagnose`` adds for it, under the same evidence and thresholds. A target named
    twice is diagnosed once. If a parameter is missing the graph cannot be compiled, and every
    target's list is empty.

    Args:
        graph: The graph.
        targets: The ids of the nodes whose weak points are wanted; may be empty.
        evidence: Node ids and their observed values; none by default.
        claim_threshold: The threshold passed to ``claims``, a gap in natural log-odds.
        failure_threshold: The threshold passed to ``single_points_of_failure``, a fraction of each
            target's own probability.
        min_supports: The smallest group passed to ``correlated_support``, at least 2.
        engine: The exact engine; variable elimination by default.

    Returns:
        The graph-wide findings, and each target's weak points in the order the targets were given;
        ``report.to_dicts()`` is ready for ``json.dumps``.

    Raises:
        ValidationError: If ``targets`` is a single string, a threshold or ``min_supports`` is out of
            range, a target or an evidence node is not an inference variable, or the evidence observes
            a target.
        CompileError: If the graph has no missing parameter but still fails to compile, for example
            over conflicting bases on merged nodes.
        ZeroProbabilityError: If the evidence and the ``exclusive`` constraints have probability zero.
    """
    if isinstance(targets, str):
        msg = f"targets must be a collection of node ids, not the string {targets!r}"
        raise ValidationError(msg)
    targets = list(dict.fromkeys(targets))
    findings = missing_parameters(graph)
    structural = [*unanchored(graph), *correlated_support(graph, min_supports)]
    if findings:
        return Report((*findings, *structural), dict.fromkeys(targets, ()))
    findings.extend(structural)
    engine = engine or VariableElimination()
    network = compile_graph(graph)
    findings.extend(claims(graph, network, evidence=evidence, threshold=claim_threshold, engine=engine))
    weak_points: dict[str, list[Finding]] = {}
    for target in targets:
        slopes = derivatives(network, target, evidence=evidence, engine=engine)
        found = weak_points[target] = []
        found.extend(sensitivity_findings(network, target, slopes, evidence=evidence))
        found.extend(crux_findings(network, target, slopes, evidence=evidence))
        run = interventions(network, target, evidence=evidence, threshold=failure_threshold, engine=engine)
        found.extend(single_point_of_failure_findings(run))
        found.extend(failure_impact_findings(run))
        found.extend(value_of_information(network, target, evidence=evidence, engine=engine))
    return Report(findings, weak_points)

failure_impact(network, target, *, evidence=None, threshold=DEFAULT_FAILURE_THRESHOLD, engine=None)

Rank the other variables Y by P(target | do(Y = false), evidence) / P(target | evidence).

The ratio is the fraction of the target's probability that survives the failure of Y, with the intervention of single_points_of_failure: below 1 the failure lowers the target, above 1 it raises it. Every candidate of single_points_of_failure is ranked, lowest ratio first, and the threshold only marks the line: a finding whose ratio falls below it, under the same rounding rule, is exactly one single_points_of_failure reports. The ranking therefore orders the premises by failure impact without a second run at another threshold, which matters because a premise behind one requires of strength r keeps at least 1 - r of the target and crosses the default line only for r > 0.9; see single_points_of_failure.

Parameters:

Name Type Description Default
network Network

The network.

required
target str

The id of the target node.

required
evidence Mapping[str, bool] | None

Node ids and their observed values; none by default.

None
threshold float

The fraction of P(target | evidence) that marks a single point of failure, as for single_points_of_failure; it does not filter the ranking.

DEFAULT_FAILURE_THRESHOLD
engine Engine | None

The exact engine; variable elimination by default.

None

Returns:

Type Description
list[Finding]

One finding per candidate, lowest ratio first, ties in network order; its value is the ratio,

list[Finding]

and its details give P(target | evidence) as baseline,

list[Finding]

P(target | do(Y = false), evidence) as p_target_if_false, the fraction as

list[Finding]

threshold, and single_point_of_failure, true for a ratio below the threshold. When

list[Finding]

P(target | evidence) is zero there is nothing to lose and the ratio is undefined, so

list[Finding]

there are no findings.

Raises:

Type Description
ValidationError

If threshold is not in [0, 1], target or an evidence node is not an inference variable, or the evidence observes the target.

ZeroProbabilityError

If the evidence and the exclusive constraints have probability zero, or the constraints alone do under one of the interventions.

Source code in src/credencegraph/diagnostics/weak_points.py
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
def failure_impact(
    network: Network,
    target: str,
    *,
    evidence: Mapping[str, bool] | None = None,
    threshold: float = DEFAULT_FAILURE_THRESHOLD,
    engine: Engine | None = None,
) -> list[Finding]:
    """Rank the other variables Y by ``P(target | do(Y = false), evidence) / P(target | evidence)``.

    The ratio is the fraction of the target's probability that survives the failure of Y, with the
    intervention of ``single_points_of_failure``: below 1 the failure lowers the target, above 1 it
    raises it. Every candidate of ``single_points_of_failure`` is ranked, lowest ratio first, and the
    threshold only marks the line: a finding whose ratio falls below it, under the same rounding
    rule, is exactly one ``single_points_of_failure`` reports. The ranking therefore orders the
    premises by failure impact without a second run at another threshold, which matters because a
    premise behind one ``requires`` of strength r keeps at least ``1 - r`` of the target and crosses
    the default line only for r > 0.9; see ``single_points_of_failure``.

    Args:
        network: The network.
        target: The id of the target node.
        evidence: Node ids and their observed values; none by default.
        threshold: The fraction of ``P(target | evidence)`` that marks a single point of failure, as
            for ``single_points_of_failure``; it does not filter the ranking.
        engine: The exact engine; variable elimination by default.

    Returns:
        One finding per candidate, lowest ratio first, ties in network order; its value is the ratio,
        and its details give ``P(target | evidence)`` as ``baseline``,
        ``P(target | do(Y = false), evidence)`` as ``p_target_if_false``, the fraction as
        ``threshold``, and ``single_point_of_failure``, true for a ratio below the threshold. When
        ``P(target | evidence)`` is zero there is nothing to lose and the ratio is undefined, so
        there are no findings.

    Raises:
        ValidationError: If ``threshold`` is not in [0, 1], ``target`` or an evidence node is not an
            inference variable, or the evidence observes the target.
        ZeroProbabilityError: If the evidence and the ``exclusive`` constraints have probability zero,
            or the constraints alone do under one of the interventions.
    """
    run = interventions(network, target, evidence=evidence, threshold=threshold, engine=engine)
    return failure_impact_findings(run)

missing_parameters(graph)

Find inference variables without a base and inferential relations without a strength.

Nodes merged by equivalent relations form one variable, which needs a base on at least one of them. A graph with any such finding cannot be compiled: there is no default credence.

Relation already refuses to be built without a strength when its type needs one, so the second check only fires on a graph whose relations were altered after validation.

Parameters:

Name Type Description Default
graph Graph

The graph.

required

Returns:

Type Description
list[Finding]

One finding per variable without a base, then one per relation without a strength.

Source code in src/credencegraph/diagnostics/structure.py
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
def missing_parameters(graph: Graph) -> list[Finding]:
    """Find inference variables without a ``base`` and inferential relations without a ``strength``.

    Nodes merged by ``equivalent`` relations form one variable, which needs a base on at least one of
    them. A graph with any such finding cannot be compiled: there is no default credence.

    ``Relation`` already refuses to be built without a strength when its type needs one, so the
    second check only fires on a graph whose relations were altered after validation.

    Args:
        graph: The graph.

    Returns:
        One finding per variable without a base, then one per relation without a strength.
    """
    findings: list[Finding] = []
    for root, ids in inference_sets(graph).items():
        if any(graph.nodes[node_id].base is not None for node_id in ids):
            continue
        findings.append(
            Finding(
                id=f"{MISSING_PARAMETER}:base:{root}",
                diagnostic=MISSING_PARAMETER,
                message=f"node {_describe(ids)} is an inference variable without a base; there is no default credence",
                nodes=tuple(ids),
            )
        )
    for relation in graph.relations.values():
        if relation.type in STRENGTH_TYPES and relation.strength is None:
            findings.append(
                Finding(
                    id=f"{MISSING_PARAMETER}:strength:{relation.id}",
                    diagnostic=MISSING_PARAMETER,
                    message=f"{relation.type} relation {relation.id!r} has no strength",
                    nodes=(relation.source, relation.target),
                    relations=(relation.id,),
                )
            )
    return findings

sensitivity(network, target, *, evidence=None, engine=None)

Report dP(target | evidence)/dθ for every parameter θ.

Parameters:

Name Type Description Default
network Network

The network.

required
target str

The id of the target node.

required
evidence Mapping[str, bool] | None

Node ids and their observed values; none by default.

None
engine Engine | None

The exact engine; variable elimination by default.

None

Returns:

Type Description
list[Finding]

One finding per parameter, largest magnitude first; its value is the signed derivative.

Raises:

Type Description
ValidationError

If target or an evidence node is not an inference variable of the network, or the evidence observes the target.

ZeroProbabilityError

If the evidence and the exclusive constraints have probability zero.

Source code in src/credencegraph/diagnostics/weak_points.py
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
def sensitivity(
    network: Network,
    target: str,
    *,
    evidence: Mapping[str, bool] | None = None,
    engine: Engine | None = None,
) -> list[Finding]:
    """Report ``dP(target | evidence)/dθ`` for every parameter θ.

    Args:
        network: The network.
        target: The id of the target node.
        evidence: Node ids and their observed values; none by default.
        engine: The exact engine; variable elimination by default.

    Returns:
        One finding per parameter, largest magnitude first; its value is the signed derivative.

    Raises:
        ValidationError: If ``target`` or an evidence node is not an inference variable of the
            network, or the evidence observes the target.
        ZeroProbabilityError: If the evidence and the ``exclusive`` constraints have probability zero.
    """
    slopes = derivatives(network, target, evidence=evidence, engine=engine)
    return sensitivity_findings(network, target, slopes, evidence=evidence)

single_points_of_failure(network, target, *, evidence=None, threshold=DEFAULT_FAILURE_THRESHOLD, engine=None)

Find the variables Y whose failure alone sinks the target: P(target | do(Y = false), evidence).

The intervention cuts the relations into Y and fixes it false, as in "suppose this premise is simply wrong"; the evidence is then conditioned on. A variable is reported when that probability is below threshold * P(target | evidence): the threshold is a fraction of the target's own probability, not a probability, so the same default separates the premises that sink the target from those that merely dent it whether the target starts at 0.9 or at 0.01. A threshold of 1 reports every variable whose failure lowers the target at all; 0 reports none.

A premise behind one requires of strength r is reported only if r > 1 - threshold. Let the only directed path from Y to T be one requires relation Y -> T of strength r, with no evidence and no exclusive constraint. P(T = 1 | parents) is Y's necessity gate, 1 - r when Y is false and 1 when it is true, times a factor A that does not involve Y. Failing Y leaves the distribution of T's other parents as it was, so P(T | do(Y = false)) = (1 - r) E[A], while P(T) <= E[A] because the gate is at most 1. Hence

P(T | do(Y = false)) / P(T) >= 1 - r,

and when Y shares no ancestor with T's other parents the ratio is exactly (1 - r) / ((1 - r) + r p) with p = P(Y), which tends to 1 - r as p tends to 1. At the default threshold of 0.1 such a premise is never reported when r <= 0.9, however probable it is: an empty list then says that no requires is stronger than 0.9, not that nothing is fatal. failure_impact ranks every candidate by the same ratio, so the premises that dent the target most are visible whether or not one crosses the line.

Observed variables are not candidates: the evidence already says whether they hold. Nor is a variable whose failure the evidence rules out, where P(evidence | do(Y = false)) is zero: given what was observed, Y did not fail.

Parameters:

Name Type Description Default
network Network

The network.

required
target str

The id of the target node.

required
evidence Mapping[str, bool] | None

Node ids and their observed values; none by default.

None
threshold float

The fraction of P(target | evidence) below which the target counts as failed; a probability within a relative 1e-12 of the cutoff counts as equal to it, so rounding cannot tip a finding however small P(target | evidence) is. The default, 0.1, an order of magnitude, is a convention chosen for this package rather than a value the model fixes.

DEFAULT_FAILURE_THRESHOLD
engine Engine | None

The exact engine; variable elimination by default.

None

Returns:

Type Description
list[Finding]

One finding per single point of failure, lowest remaining probability first; its value is

list[Finding]

P(target | do(Y = false), evidence), and its details give P(target | evidence) as

list[Finding]

baseline and the fraction as threshold.

Raises:

Type Description
ValidationError

If threshold is not in [0, 1], target or an evidence node is not an inference variable, or the evidence observes the target.

ZeroProbabilityError

If the evidence and the exclusive constraints have probability zero, or the constraints alone do under one of the interventions.

Source code in src/credencegraph/diagnostics/weak_points.py
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
def single_points_of_failure(
    network: Network,
    target: str,
    *,
    evidence: Mapping[str, bool] | None = None,
    threshold: float = DEFAULT_FAILURE_THRESHOLD,
    engine: Engine | None = None,
) -> list[Finding]:
    """Find the variables Y whose failure alone sinks the target: ``P(target | do(Y = false), evidence)``.

    The intervention cuts the relations into Y and fixes it false, as in "suppose this premise is
    simply wrong"; the evidence is then conditioned on. A variable is reported when that probability
    is below ``threshold * P(target | evidence)``: the threshold is a fraction of the target's own
    probability, not a probability, so the same default separates the premises that sink the target
    from those that merely dent it whether the target starts at 0.9 or at 0.01. A threshold of 1
    reports every variable whose failure lowers the target at all; 0 reports none.

    **A premise behind one ``requires`` of strength r is reported only if r > 1 - threshold.** Let
    the only directed path from Y to T be one ``requires`` relation Y -> T of strength r, with no
    evidence and no ``exclusive`` constraint. ``P(T = 1 | parents)`` is Y's necessity gate, 1 - r
    when Y is false and 1 when it is true, times a factor A that does not involve Y. Failing Y leaves
    the distribution of T's other parents as it was, so ``P(T | do(Y = false)) = (1 - r) E[A]``,
    while ``P(T) <= E[A]`` because the gate is at most 1. Hence

        P(T | do(Y = false)) / P(T) >= 1 - r,

    and when Y shares no ancestor with T's other parents the ratio is exactly
    ``(1 - r) / ((1 - r) + r p)`` with ``p = P(Y)``, which tends to 1 - r as p tends to 1. At the
    default threshold of 0.1 such a premise is never reported when r <= 0.9, however probable it is:
    an empty list then says that no ``requires`` is stronger than 0.9, not that nothing is fatal.
    ``failure_impact`` ranks every candidate by the same ratio, so the premises that dent the target
    most are visible whether or not one crosses the line.

    Observed variables are not candidates: the evidence already says whether they hold. Nor is a
    variable whose failure the evidence rules out, where ``P(evidence | do(Y = false))`` is zero:
    given what was observed, Y did not fail.

    Args:
        network: The network.
        target: The id of the target node.
        evidence: Node ids and their observed values; none by default.
        threshold: The fraction of ``P(target | evidence)`` below which the target counts as failed;
            a probability within a relative 1e-12 of the cutoff counts as equal to it, so rounding
            cannot tip a finding however small ``P(target | evidence)`` is. The default, 0.1, an
            order of magnitude, is a convention chosen for this package rather than a value the model
            fixes.
        engine: The exact engine; variable elimination by default.

    Returns:
        One finding per single point of failure, lowest remaining probability first; its value is
        ``P(target | do(Y = false), evidence)``, and its details give ``P(target | evidence)`` as
        ``baseline`` and the fraction as ``threshold``.

    Raises:
        ValidationError: If ``threshold`` is not in [0, 1], ``target`` or an evidence node is not an
            inference variable, or the evidence observes the target.
        ZeroProbabilityError: If the evidence and the ``exclusive`` constraints have probability zero,
            or the constraints alone do under one of the interventions.
    """
    run = interventions(network, target, evidence=evidence, threshold=threshold, engine=engine)
    return single_point_of_failure_findings(run)

sorries(graph, target, directory, *, engine=None)

Report the steps a target depends on that remain sorries, ranked by the target's sensitivity.

Each sorry with a strength is ranked by dP(target)/d strength, taken exactly as in derivatives with every parameter at its credence mean. A discharged step is not ranked: credence applies only to what remains unproven.

Parameters:

Name Type Description Default
graph Graph

The graph, which must compile for inference.

required
target str

The target node's id.

required
directory str | Path

The directory script paths are relative to.

required
engine Engine | None

The exact engine; variable elimination by default.

None

Returns:

Type Description
SorryReport

The report.

Raises:

Type Description
CompileError

If the graph cannot be compiled.

ValidationError

If target is not an inference variable.

ZeroProbabilityError

If no world satisfies the exclusive constraints.

Source code in src/credencegraph/diagnostics/obligations.py
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
def sorries(graph: Graph, target: str, directory: str | Path, *, engine: Engine | None = None) -> SorryReport:
    """Report the steps a target depends on that remain sorries, ranked by the target's sensitivity.

    Each sorry with a strength is ranked by ``dP(target)/d strength``, taken exactly as in
    ``derivatives`` with every parameter at its credence mean. A discharged step is not ranked:
    credence applies only to what remains unproven.

    Args:
        graph: The graph, which must compile for inference.
        target: The target node's id.
        directory: The directory script paths are relative to.
        engine: The exact engine; variable elimination by default.

    Returns:
        The report.

    Raises:
        CompileError: If the graph cannot be compiled.
        ValidationError: If ``target`` is not an inference variable.
        ZeroProbabilityError: If no world satisfies the ``exclusive`` constraints.
    """
    slopes = derivatives(compile_graph(graph), target, engine=engine)
    steps = dependencies(graph, target)
    discharged = []
    found = []
    for relation in steps:
        reason = sorry_reason(relation, directory)
        if reason is None:
            discharged.append(relation.id)
            continue
        found.append(
            Sorry(
                relation.id,
                relation.type,
                relation.source,
                relation.target,
                reason,
                relation.obligation and relation.obligation.statement,
                slopes.get(ParameterKey("strength", relation.id)),
            )
        )
    found.sort(key=lambda sorry: (sorry.sensitivity is None, -abs(sorry.sensitivity or 0.0)))
    return SorryReport(target, tuple(relation.id for relation in steps), tuple(discharged), tuple(found))

unanchored(graph)

Find inference variables with no source and no inferential parents.

Such a variable is an assumption nobody wrote down: nothing in the graph says where its credence comes from. Nodes merged by equivalent relations are one variable, anchored if any of them has a source or is the target of a requires, supports or refutes relation.

Parameters:

Name Type Description Default
graph Graph

The graph.

required

Returns:

Type Description
list[Finding]

One finding per unanchored variable, in insertion order.

Source code in src/credencegraph/diagnostics/structure.py
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
def unanchored(graph: Graph) -> list[Finding]:
    """Find inference variables with no source and no inferential parents.

    Such a variable is an assumption nobody wrote down: nothing in the graph says where its credence
    comes from. Nodes merged by ``equivalent`` relations are one variable, anchored if any of them
    has a source or is the target of a ``requires``, ``supports`` or ``refutes`` relation.

    Args:
        graph: The graph.

    Returns:
        One finding per unanchored variable, in insertion order.
    """
    has_parents = {relation.target for relation in graph.relations.values() if relation.type in STRENGTH_TYPES}
    findings: list[Finding] = []
    for root, ids in inference_sets(graph).items():
        if any(graph.nodes[node_id].sources or node_id in has_parents for node_id in ids):
            continue
        findings.append(
            Finding(
                id=f"{UNANCHORED}:{root}",
                diagnostic=UNANCHORED,
                message=f"node {_describe(ids)} has no source and no inferential parents: an assumption nobody wrote down",
                nodes=tuple(ids),
            )
        )
    return findings

value_of_information(network, target, *, evidence=None, engine=None)

Rank the other variables Y by the mutual information I(target; Y | evidence), in bits.

I(T; Y | E) is the expected reduction in the entropy of T from learning whether Y is true, once E is known, so the top of the ranking is what would most change the target if it were resolved next, and is the first thing to investigate. It is at most 1 bit, the entropy of a binary variable. Observed variables are left out: learning them again tells nothing.

The unit, bits (logarithms to base 2), is a choice of this package; the ranking does not depend on it. Multiply by ln 2 for nats.

Parameters:

Name Type Description Default
network Network

The network.

required
target str

The id of the target node.

required
evidence Mapping[str, bool] | None

Node ids and their observed values; none by default.

None
engine Engine | None

The exact engine; variable elimination by default.

None

Returns:

Type Description
list[Finding]

One finding per other unobserved proposition variable, most informative first. Its details

list[Finding]

give P(target | evidence) as p_target and, for each value of Y with positive

list[Finding]

probability, P(target | Y, evidence) as p_target_given_true or _false.

Raises:

Type Description
ValidationError

If target or an evidence node is not an inference variable of the network, or the evidence observes the target.

ZeroProbabilityError

If the evidence and the exclusive constraints have probability zero.

Source code in src/credencegraph/diagnostics/weak_points.py
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
def value_of_information(
    network: Network,
    target: str,
    *,
    evidence: Mapping[str, bool] | None = None,
    engine: Engine | None = None,
) -> list[Finding]:
    """Rank the other variables Y by the mutual information ``I(target; Y | evidence)``, in bits.

    ``I(T; Y | E)`` is the expected reduction in the entropy of T from learning whether Y is true,
    once E is known, so the top of the ranking is what would most change the target if it were
    resolved next, and is the first thing to investigate. It is at most 1 bit, the entropy of a
    binary variable. Observed variables are left out: learning them again tells nothing.

    The unit, bits (logarithms to base 2), is a choice of this package; the ranking does not depend
    on it. Multiply by ``ln 2`` for nats.

    Args:
        network: The network.
        target: The id of the target node.
        evidence: Node ids and their observed values; none by default.
        engine: The exact engine; variable elimination by default.

    Returns:
        One finding per other unobserved proposition variable, most informative first. Its details
        give ``P(target | evidence)`` as ``p_target`` and, for each value of Y with positive
        probability, ``P(target | Y, evidence)`` as ``p_target_given_true`` or ``_false``.

    Raises:
        ValidationError: If ``target`` or an evidence node is not an inference variable of the
            network, or the evidence observes the target.
        ZeroProbabilityError: If the evidence and the ``exclusive`` constraints have probability zero.
    """
    engine = engine or VariableElimination()
    given = _observed(network, target, evidence).evidence
    findings = []
    for name, members in _others(network, target, given):
        table = [
            [engine.query(network, Query({target: bool(t), name: bool(y)}, given)) for y in (0, 1)] for t in (0, 1)
        ]
        bits = _mutual_information(table)
        details = {"p_target": table[1][0] + table[1][1]}
        for y, label in ((1, "true"), (0, "false")):
            p_y = table[0][y] + table[1][y]
            if p_y > 0.0:
                details[f"p_target_given_{label}"] = table[1][y] / p_y
        findings.append(
            Finding(
                id=f"{VALUE_OF_INFORMATION}:{name}",
                diagnostic=VALUE_OF_INFORMATION,
                message=(
                    f"learning whether {name!r} is true would give {bits:.3g} bits of information about "
                    f"{target!r}{' given ' + render_evidence(given) if given else ''}"
                ),
                nodes=members,
                value=bits,
                details=details,
            )
        )
    return sorted(findings, key=lambda finding: -(finding.value or 0.0))

verify(graph, directory, *, timeout=DEFAULT_TIMEOUT)

Re-check every discharge in a graph.

A script discharge is run with the current interpreter, from the script's directory, and is flagged when the script cannot be read, was never run, changed since its recorded run, has no reachable way to fail (see can_fail), or now fails: as contradicted when its recorded run passed, or when it now passes though its recorded run failed, and as check-failed otherwise. When a numerical script exits 0, the last non-blank line it printed is its result, flagged as result-missing when it is not a number and as out-of-tolerance when it lies further than tolerance from value. A citation is flagged when its quote is not in the cited document, read as a file relative to directory, or its digest is not the quote's; a citation of anything else is unchecked. A formal discharge is unchecked, since no proof checker ships with this package.

Parameters:

Name Type Description Default
graph Graph

The graph.

required
directory str | Path

The directory that script paths and cited documents are relative to, normally the graph file's.

required
timeout float

Seconds a script may run before it is stopped and its run counted as failed.

DEFAULT_TIMEOUT

Returns:

Type Description
list[Check]

One check per discharge, in the graph's relation order.

Source code in src/credencegraph/diagnostics/obligations.py
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
def verify(graph: Graph, directory: str | Path, *, timeout: float = DEFAULT_TIMEOUT) -> list[Check]:
    """Re-check every discharge in a graph.

    A script discharge is run with the current interpreter, from the script's directory, and is
    flagged when the script cannot be read, was never run, changed since its recorded run, has no
    reachable way to fail (see ``can_fail``), or now fails: as ``contradicted`` when its recorded run
    passed, or when it now passes though its recorded run failed, and as ``check-failed`` otherwise.
    When a numerical script exits 0, the last non-blank line it printed is its result, flagged as
    ``result-missing`` when it is not a number and as ``out-of-tolerance`` when it lies further than
    ``tolerance`` from ``value``. A citation is flagged when its
    quote is not in the cited document, read as a file relative to ``directory``, or its digest is
    not the quote's; a citation of anything else is unchecked. A ``formal`` discharge is unchecked,
    since no proof checker ships with this package.

    Args:
        graph: The graph.
        directory: The directory that script paths and cited documents are relative to, normally
            the graph file's.
        timeout: Seconds a script may run before it is stopped and its run counted as failed.

    Returns:
        One check per discharge, in the graph's relation order.
    """
    directory = Path(directory)
    checks = []
    for relation in graph.relations.values():
        discharge = relation.obligation and relation.obligation.discharge
        if discharge is None:
            continue
        if discharge.kind in SCRIPT_KINDS:
            checks.append(_check_script(relation, discharge, directory, timeout))
        elif discharge.kind == CITATION:
            checks.append(_check_citation(relation, discharge, directory))
        elif discharge.kind == FORMAL:
            reason = f"no proof checker ships with this package to build theorem {discharge.theorem!r}"
            checks.append(Check(relation.id, discharge.kind, UNCHECKED, reason=reason))
        else:
            checks.append(Check(relation.id, discharge.kind, NOT_CHECKABLE))
    return checks

credencegraph.diagnostics.claims.claims(graph, network=None, *, evidence=None, threshold=DEFAULT_CLAIM_THRESHOLD, engine=None)

Compare each node's stated credence with the probability its premises give it.

The computed value is the point answer P(X | evidence) of the network, P(X) when there is no evidence. A node whose variable is observed is skipped: its value is then the observation, not something its premises deliver. The two are compared in log-odds, logit(p) = ln(p / (1 - p)), so that a gap counts by the factor between the odds rather than by the difference between the probabilities: 0.07 against 0.001 is far apart, 0.5 against 0.569 is not. When the stated mean exceeds the computed value by more than threshold the text asserts more confidence than its own argument delivers (an overclaim); when it falls short by more than threshold, less (an underclaim). A stated or computed value of exactly 0 or 1 is infinitely far from any other. Nodes with a stated credence that are not inference variables have nothing to compare against and are skipped.

Parameters:

Name Type Description Default
graph Graph

The graph.

required
network Network | None

The network compiled from graph; compiled here when omitted.

None
evidence Mapping[str, bool] | None

Node ids and their observed values; none by default.

None
threshold float

The largest gap that is not reported, in natural log-odds: ln(k) reports odds that differ by more than a factor of k. A gap within 1e-12 of it counts as equal to it, so rounding cannot tip a finding either way. The default, 2 ln(11 / 9), about 0.401, is a convention chosen for this package rather than a value the model fixes: it is the log-odds gap between 0.45 and 0.55, the infimum of the gaps between probabilities more than 0.1 apart, which such pairs approach arbitrarily closely but never reach. With the same 1e-12 allowance, every pair more than 0.1 + 1e-12 apart in probability, which is every pair an absolute threshold of 0.1 would report, is reported.

DEFAULT_CLAIM_THRESHOLD
engine Engine | None

The inference engine; variable elimination by default.

None

Returns:

Type Description
list[Finding]

One finding per node whose stated and computed credences differ by more than threshold,

list[Finding]

in insertion order. Its value is the signed gap in probability, stated minus computed, which

list[Finding]

stays finite when the log-odds gap is not. Its details are stated and computed, in

list[Finding]

probability, and threshold_log_odds, the threshold applied, in natural log-odds.

Raises:

Type Description
ValidationError

If threshold is not a finite number >= 0, or an evidence node is not an inference variable.

CompileError

If network is omitted and graph does not compile.

ZeroProbabilityError

If the evidence and the exclusive constraints have probability zero.

Source code in src/credencegraph/diagnostics/claims.py
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
def claims(
    graph: Graph,
    network: Network | None = None,
    *,
    evidence: Mapping[str, bool] | None = None,
    threshold: float = DEFAULT_CLAIM_THRESHOLD,
    engine: Engine | None = None,
) -> list[Finding]:
    """Compare each node's ``stated`` credence with the probability its premises give it.

    The computed value is the point answer ``P(X | evidence)`` of the network, ``P(X)`` when there
    is no evidence. A node whose variable is observed is skipped: its value is then the observation,
    not something its premises deliver. The two are compared in
    log-odds, ``logit(p) = ln(p / (1 - p))``, so that a gap counts by the factor between the odds
    rather than by the difference between the probabilities: 0.07 against 0.001 is far apart, 0.5
    against 0.569 is not. When the stated mean exceeds the computed value by more than ``threshold``
    the text asserts more confidence than its own argument delivers (an overclaim); when it falls
    short by more than ``threshold``, less (an underclaim). A stated or computed value of exactly 0
    or 1 is infinitely far from any other. Nodes with a ``stated`` credence that are not inference
    variables have nothing to compare against and are skipped.

    Args:
        graph: The graph.
        network: The network compiled from ``graph``; compiled here when omitted.
        evidence: Node ids and their observed values; none by default.
        threshold: The largest gap that is not reported, in natural log-odds: ``ln(k)`` reports
            odds that differ by more than a factor of ``k``. A gap within 1e-12 of it counts as equal
            to it, so rounding cannot tip a finding either way. The default, ``2 ln(11 / 9)``, about
            0.401, is a convention chosen for this package rather than a value the model fixes: it
            is the log-odds gap between 0.45 and 0.55, the infimum of the gaps between probabilities
            more than 0.1 apart, which such pairs approach arbitrarily closely but never reach. With
            the same 1e-12 allowance, every pair more than 0.1 + 1e-12 apart in probability, which
            is every pair an absolute threshold of 0.1 would report, is reported.
        engine: The inference engine; variable elimination by default.

    Returns:
        One finding per node whose stated and computed credences differ by more than ``threshold``,
        in insertion order. Its value is the signed gap in probability, stated minus computed, which
        stays finite when the log-odds gap is not. Its details are ``stated`` and ``computed``, in
        probability, and ``threshold_log_odds``, the threshold applied, in natural log-odds.

    Raises:
        ValidationError: If ``threshold`` is not a finite number >= 0, or an evidence node is not an
            inference variable.
        CompileError: If ``network`` is omitted and ``graph`` does not compile.
        ZeroProbabilityError: If the evidence and the ``exclusive`` constraints have probability zero.
    """
    limit = _claim_threshold(threshold)
    network = network if network is not None else compile_graph(graph)
    engine = engine or VariableElimination()
    given = Query({}, evidence or {}).evidence
    observed = {network.index(node_id) for node_id in given}
    if given:
        engine.query(network, Query({}, given))  # raises when the evidence has probability zero
    variables = {member for variable in network.variables for member in variable.members}
    suffix = f" given {render_evidence(given)}" if given else ""
    findings: list[Finding] = []
    for node in graph:
        if node.stated is None or node.id not in variables or network.index(node.id) in observed:
            continue
        stated = node.stated.mean
        computed = engine.query(network, Query({node.id: True}, given))
        gap = stated - computed
        if abs(_log_odds_gap(stated, computed)) <= limit + ROUNDING:
            continue
        diagnostic, verb = (OVERCLAIM, "more") if gap > 0 else (UNDERCLAIM, "less")
        findings.append(
            Finding(
                id=f"{diagnostic}:{node.id}",
                diagnostic=diagnostic,
                message=(
                    f"node {node.id!r} is stated at {stated:.3g} but its premises give {computed:.3g}{suffix}: "
                    f"the text asserts {abs(gap):.3g} {verb} confidence than its argument delivers"
                ),
                nodes=(node.id,),
                value=gap,
                details={"stated": stated, "computed": computed, "threshold_log_odds": limit},
            )
        )
    return findings