Package 'alethiaR'

Align messy entity strings against a reference list

Description

The main entry point, mirroring the Python 'alethia()'. The pipeline is:

Usage

alethia(
  dirty_entries,
  reference_entries,
  model = "rapidfuzz",
  threshold = NULL,
  use_exact_matching = TRUE,
  exact_match_case_sensitive = FALSE,
  drop_duplicates = TRUE
)

Arguments

dirty_entries	Character vector of messy strings.
reference_entries	Character vector of canonical strings.
model	'"rapidfuzz"' for token-sort string similarity (the default, and the only backend needing no model), or anything [as_embedder()] accepts.
threshold	Minimum score to accept a match; below it the entry is reported as no match. 'NULL' means 'DEFAULT_THRESHOLD' for embeddings and no threshold for string matching. Scores are on '[0, 1]' either way, but string similarity and cosine measure different things, so a value tuned against one is not calibrated for the other.
use_exact_matching	Short-circuit entries already present in the references.
exact_match_case_sensitive	Whether that short-circuit respects case.
drop_duplicates	Collapse identical result rows, giving one row per distinct input. 'FALSE' keeps the input length and order, for joining back onto a table.

Details

1. drop 'NA' and null-like entries, remembering their positions; 2. short-circuit entries that already appear in the reference list; 3. send whatever remains to the chosen backend; 4. splice all three groups back into the caller's original row order.

Value

A data frame carrying 'given_entity', 'alethia_prediction', 'alethia_score' and 'alethia_method', in input order.

Examples

alethia(
  c("Bombay", "Calcutta", "Mumbai"),
  c("Mumbai", "Kolkata", "Chennai")
)

# Entries already present in the references short-circuit as exact matches, and
# null-like values are preserved as no-match rows rather than dropped.
alethia(c("Mumbai", NA, "N/A"), c("Mumbai"), drop_duplicates = FALSE)

---

Match composite entries by splitting them on a separator first

Description

Mirrors the Python 'alethia_split()'. Each entry is split into pieces on a literal separator, every piece is matched independently against 'reference_entries' via [alethia()], and the highest-scoring piece becomes the entry's prediction. Ties break on first occurrence, same as the rest of the package. Every piece's own match rides along in 'alethia_parts'.

Usage

alethia_split(dirty_entries, reference_entries, split_on, ...)

Arguments

dirty_entries	Character vector of messy strings.
reference_entries	Character vector of canonical strings.
split_on	Literal separator each entry is split on (e.g. '","', '";"', '" and "'). Entries with no separator, or that split to nothing usable, are matched whole.
...	Forwarded to [alethia()] for the per-piece matching ('model', 'threshold', ...).

Value

A data frame of 'given_entity', 'alethia_prediction', 'alethia_score', 'alethia_matched_part' (the winning piece), and 'alethia_parts' (a list column: every piece tried, as a list of 'part'/'prediction'/'score', best first).

Examples

alethia_split(
  c("CNS Tuberculosis, Septic Shock"),
  c("Tuberculosis of nervous system", "Septic shock"),
  split_on = ","
)

---

Alignment loss

Description

Mean '||x - y||^alpha' over matched positive pairs, on L2-normalised embeddings. Lower means the model keeps a string and its dirtied variant close. After Wang & Isola (2020).

Usage

alignment_loss(src_emb, var_emb, alpha = 2)

Arguments

src_emb, var_emb	Matched embedding matrices, row 'i' a positive pair.
alpha	Exponent on the distance.

Value

Scalar loss.

See Also

Other assess metrics: centered_separability(), hubness(), intrinsic_dimensionality(), mutual_nn_rate(), positive_pair_rank(), reference_separability(), retrieval_margin(), uniformity_loss()

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
refs <- bag(c("aspirin", "ibuprofen", "paracetamol", "metformin"))
queries <- bag(c("asprin", "ibuprofn", "paracetmol", "metfornin"))
alignment_loss(refs, queries)

---

Blank out matches scoring below a threshold

Description

Sub-threshold rows are reported as no match. 'NULL' leaves the frame untouched.

Usage

apply_threshold(results, threshold)

Arguments

results	Data frame with 'alethia_prediction' and 'alethia_score'.
threshold	Minimum acceptable score, or 'NULL'.

Value

The frame, with sub-threshold rows set to 'NA'.

Examples

res <- match_by_strings("Bombay", c("Mumbai", "Kolkata"))
apply_threshold(res, 0.9)

---

Coerce a model specification into an embedder

Description

Mirrors 'alethia.embedder.as_embedder'. An embedder is any object with an 'encode(texts)' method returning an 'n x d' numeric matrix. This function accepts:

Usage

as_embedder(model, name = NULL)

Arguments

model	A function, an embedder, or a model name.
name	Optional display name.

Details

* a function 'f(texts) -> matrix', wrapped as a callable embedder; * an object already carrying an 'encode' element (returned unchanged); * a character model name, resolved through the registered backend (see [set_embedding_backend()]).

Every embedding source sits behind this seam, so matching, clustering and assessment share one implementation.

Value

An object of class 'alethia_embedder'.

Examples

# Any function from character vector to matrix is an embedder.
bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
e <- as_embedder(bag, name = "letter-count")
e

---

Score candidate embedding models on your own data, without labels

Description

Each model is scored on the metrics in [assess-metrics], z-scored **across models** and summed. The composite therefore needs at least two models; with one the score is 'NA' and only the component metrics are reported. Weights are normalised **within metric family**, so correlated views of one property cannot count it twice, and a family that could not be computed drops out.

Usage

assess_models(
  queries,
  references,
  models,
  weights = NULL,
  family_normalize = TRUE,
  variants = NULL
)

DEFAULT_METRIC_WEIGHTS

Arguments

queries	Character vector of dirty/query strings. May be empty, in which case the retrieval family is skipped and the reference geometry alone is scored.
references	Character vector of canonical strings.
models	Named list of model specifications, each anything [as_embedder()] accepts. Names are used for reporting.
weights	Optional named numeric vector overriding the per-metric weights. Interpreted within family when 'family_normalize' is 'TRUE'.
family_normalize	Rescale weights so each family contributes at most its family weight. 'FALSE' gives flat per-metric weighting.
variants	Optional list of 'source' and 'variant' character vectors, row 'i' a (clean, dirtied) pair, which enables the robustness family. Generate them with 'alethia.assess.generate_positive_pairs'; omitted, the family drops out.

Value

An object of class 'alethia_assessment' with '$table' (a data frame, best first) and '$best' (the winning model name, or 'NA' when not comparative).

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
rev_bag <- function(texts) {
  bag(vapply(texts, function(s) {
    paste(rev(strsplit(s, "")[[1]]), collapse = "")
  }, character(1)))
}
refs <- c("aspirin", "ibuprofen", "paracetamol", "metformin", "atenolol")
res <- assess_models(
  c("asprin", "ibuprofn"), refs,
  list(bag = bag, reversed = rev_bag)
)
res$best

---

Label-free geometric metrics for an embedding set

Description

These score an embedding model on your own data without ground-truth labels, by measuring the geometry it produces. Ports of 'alethia.assess.metrics'; each formula is stated so the two implementations can be checked against one another.

---

Anisotropy-corrected separability

Description

[reference_separability()] after removing the set mean, which strips the shared direction responsible for uniformly high cosines.

Usage

centered_separability(ref_emb)

Arguments

ref_emb

Numeric matrix of reference embeddings.

Value

Named list of 'centered_nn_similarity' and 'nn_margin_z'.

See Also

Other assess metrics: alignment_loss(), hubness(), intrinsic_dimensionality(), mutual_nn_rate(), positive_pair_rank(), reference_separability(), retrieval_margin(), uniformity_loss()

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
refs <- bag(c("aspirin", "ibuprofen", "paracetamol", "metformin"))
queries <- bag(c("asprin", "ibuprofn", "paracetmol", "metfornin"))
centered_separability(refs)

---

Group entity strings into clusters of duplicates

Description

Mirrors 'alethia.cluster.cluster_entities'. Edges require *mutual* nearest-neighbourhood above a cosine floor and each carries a confidence, so no hub chains the whole set into one cluster the way a plain threshold does.

Usage

cluster_entities(
  entities,
  model,
  floor = 0.8,
  k = 5,
  require_mutual = TRUE,
  min_confidence = NULL,
  canonical = c("shortest", "first")
)

Arguments

entities	Character vector to group. De-duplicated on exact text first.
model	Anything [as_embedder()] accepts.
floor	Minimum cosine for an edge.
k	Neighbourhood size for the mutual-NN test. Clamped to '[1, n - 1]'.
require_mutual	Require each entity to be in the other's top-k.
min_confidence	Optional cutoff applied to edge confidence before clustering.
canonical	How to name each cluster: '"shortest"' member or '"first"' seen.

Value

An object of class 'alethia_clusters'.

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
res <- cluster_entities(c("color", "colour", "zebra"), bag, floor = 0.8)
res
cluster_records(res)

---

Cluster membership as a tidy data frame

Description

Cluster membership as a tidy data frame

Usage

cluster_records(x)

Arguments

x	An 'alethia_clusters' object.

Value

A data frame with 'entity', 'cluster' and 'canonical' columns.

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
cluster_records(cluster_entities(c("color", "colour"), bag, floor = 0.8))

---

Default minimum cosine similarity for embedding backends

Description

String matching does not use this; a cosine and a surface-overlap score of the same value do not mean the same thing.

Usage

DEFAULT_THRESHOLD

---

Encode text with an embedder

Description

Encode text with an embedder

Usage

encode(embedder, texts)

Arguments

embedder	An 'alethia_embedder', from [as_embedder()].
texts	Character vector.

Value

Numeric matrix with one row per input.

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
dim(encode(as_embedder(bag), c("colour", "color")))

---

Hubness

Description

Skewness of the k-occurrence distribution: how unevenly references are drawn as top-k neighbours. High values are pathological, a few "hub" references absorbing many queries. After Radovanovic et al. (2010), JMLR 11:2487-2531.

Usage

hubness(query_emb, ref_emb, k = 5)

Arguments

query_emb, ref_emb	Numeric embedding matrices.
k	Neighbourhood size.

Value

Named list with 'hubness_skew'.

See Also

Other assess metrics: alignment_loss(), centered_separability(), intrinsic_dimensionality(), mutual_nn_rate(), positive_pair_rank(), reference_separability(), retrieval_margin(), uniformity_loss()

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
refs <- bag(c("aspirin", "ibuprofen", "paracetamol", "metformin"))
queries <- bag(c("asprin", "ibuprofn", "paracetmol", "metfornin"))
hubness(queries, refs, k = 2)

---

Intrinsic dimensionality

Description

Participation ratio '(sum(lambda))^2 / sum(lambda^2)' over the PCA eigenvalues, and the same divided by the ambient dimension. Higher means the model spreads the data over more directions.

Usage

intrinsic_dimensionality(emb)

Arguments

emb	Numeric matrix of embeddings.

Value

Named list of 'participation_ratio' and 'normalized_pr'.

See Also

Other assess metrics: alignment_loss(), centered_separability(), hubness(), mutual_nn_rate(), positive_pair_rank(), reference_separability(), retrieval_margin(), uniformity_loss()

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
refs <- bag(c("aspirin", "ibuprofen", "paracetamol", "metformin"))
queries <- bag(c("asprin", "ibuprofn", "paracetmol", "metfornin"))
intrinsic_dimensionality(refs)

---

Row-wise L2 normalisation

Description

The single canonical normalisation used by matching, clustering and assessment, mirroring 'alethia.embedder.l2_normalize'. Zero rows stay zero, never producing 'NaN'.

Usage

l2_normalize(x)

Arguments

x	Numeric matrix, one embedding per row.

Value

Matrix of the same shape with unit-norm rows.

Examples

l2_normalize(rbind(c(3, 4), c(0, 0)))

---

Match queries to references by embedding cosine similarity

Description

The single embedding-based matching primitive, mirroring 'alethia.embedder.match_by_embeddings'. Every embedding backend routes through it.

Usage

match_by_embeddings(queries, references, embedder, threshold = NULL)

Arguments

queries	Character vector of strings to match.
references	Character vector of candidate strings.
embedder	Anything [as_embedder()] accepts.
threshold	Optional minimum cosine. Matches below it become 'NA'.

Value

A data frame with 'given_entity', 'alethia_prediction', 'alethia_score'.

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
match_by_embeddings(c("colour"), c("color", "zebra"), bag)

---

Match strings against a reference list by token-sort similarity

Description

The counterpart of Python's 'run_rapidfuzz_matching()': for each query, the highest-scoring reference.

Usage

match_by_strings(queries, references, threshold = NULL)

Arguments

queries	Character vector of strings to match.
references	Character vector of candidate strings.
threshold	Minimum score in '[0, 1]'; below it an entry is 'NA'. 'NULL' returns the best match whatever it scores.

Value

A data frame with 'given_entity', 'alethia_prediction' and 'alethia_score'. Scores are rescaled to '[0, 1]'.

Examples

match_by_strings(
  c("Bombay", "Calcutta"),
  c("Mumbai", "Kolkata", "Chennai")
)

# Demand a close match, and report anything weaker as no match.
match_by_strings("Bombay", c("Mumbai", "Kolkata"), threshold = 0.9)

---

Candidate merge edges from mutual nearest neighbours

Description

Candidate merge edges from mutual nearest neighbours

Usage

mutual_nn_edges(emb, floor = 0.8, k = 5, require_mutual = TRUE)

Arguments

emb	Numeric matrix, one embedding per row. L2-normalised internally.
floor	Minimum cosine similarity for an edge.
k	Neighbourhood size.
require_mutual	Keep only reciprocated edges.

Value

A data frame of edges with 'i', 'j', 'cosine', 'margin_i', 'margin_j', 'mutual' and 'confidence', sorted by decreasing confidence.

Examples

emb <- rbind(c(1, 0), c(0.99, 0.14), c(0, 1))
mutual_nn_edges(emb, floor = 0.5, k = 1)

---

Mutual nearest-neighbour rate

Description

Fraction of query-to-reference top-1 assignments reciprocated within the reference's own top-k. Higher means more stable, more trustworthy matches.

Usage

mutual_nn_rate(query_emb, ref_emb, k = 5)

Arguments

query_emb, ref_emb	Numeric embedding matrices.
k	Neighbourhood size on the reference side.

Value

Scalar rate.

See Also

Other assess metrics: alignment_loss(), centered_separability(), hubness(), intrinsic_dimensionality(), positive_pair_rank(), reference_separability(), retrieval_margin(), uniformity_loss()

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
refs <- bag(c("aspirin", "ibuprofen", "paracetamol", "metformin"))
queries <- bag(c("asprin", "ibuprofn", "paracetmol", "metfornin"))
mutual_nn_rate(queries, refs, k = 2)

---

Number of clusters found

Description

Number of clusters found

Usage

n_clusters(x)

Arguments

x	An 'alethia_clusters' object.

Value

Integer count of distinct clusters.

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
n_clusters(cluster_entities(c("color", "colour", "zebra"), bag, floor = 0.8))

---

Is the local ONNX stack ready to use?

Description

Checks that the three R packages are installed and that ONNX Runtime itself has been downloaded. The runtime is a separate one-time download, not an R dependency, so installing the packages is not enough.

Usage

onnx_available()

Value

'TRUE' when local ONNX inference will work.

Examples

onnx_available()

---

Build an embedder backed by a local ONNX model

Description

Runs a sentence-embedding model on the CPU with no Python installation, using 'onnxr' for inference, 'tok' for tokenisation, and 'hfhub' to fetch the weights.

Usage

onnx_embedder(
  model_id,
  backend = c("cpu", "coreml", "cuda", "xnnpack", "openvino"),
  threads = NULL,
  max_length = 256L,
  batch_size = 32L,
  onnx_file = "onnx/model.onnx",
  tokenizer_file = "tokenizer.json"
)

Arguments

model_id	Hugging Face model id, e.g. '"sentence-transformers/all-MiniLM-L6-v2"'.
backend	Execution provider. '"cpu"' by default; the others are only useful if you have the matching hardware and an ONNX Runtime built for it.
threads	Inference threads. 'NULL' leaves it to ONNX Runtime.
max_length	Sequence length above which inputs are truncated.
batch_size	Texts per forward pass. Lower this if memory is tight; it changes speed and memory, not results.
onnx_file	Path within the repository to the ONNX weights.
tokenizer_file	Path within the repository to the tokeniser.

Details

Pooling reproduces sentence-transformers (an attention-masked mean over token vectors, then L2 normalisation), so embeddings agree with the Python package to floating-point tolerance for the same model.

The model must publish ONNX weights. Most sentence-transformers models do, under 'onnx/model.onnx'; the 'onnx-community' and 'Xenova' organisations on Hugging Face publish conversions of many others.

Value

An 'alethia_embedder', usable anywhere a model is accepted.

See Also

[use_onnx_backend()] to resolve bare model names through this path.

Examples

# Downloads ~90 MB on first use, then caches.
if (onnx_available()) {
  embedder <- onnx_embedder("sentence-transformers/all-MiniLM-L6-v2")
  alethia(c("Bombay", "Calcutta"), c("Mumbai", "Kolkata"), model = embedder)
}

---

Positive-pair rank

Description

How highly each synthesized positive ranks among *all* variants. 'src_emb[i]' and 'var_emb[i]' are a positive pair; for each source this is the percentile rank of its own variant among every variant, averaged. Higher is better. Ties take their mid-rank.

Usage

positive_pair_rank(src_emb, var_emb)

Arguments

src_emb, var_emb

Matched embedding matrices, row 'i' a positive pair.

Details

Unlike [alignment_loss()], this uses only the *ordering* of each source's similarities, so it is invariant to any strictly increasing transformation of cosine, including the affine shift from adding a common cone direction and renormalising. Hence its weight in the composite.

Value

Scalar mean percentile rank in '[0, 1]'.

See Also

Other assess metrics: alignment_loss(), centered_separability(), hubness(), intrinsic_dimensionality(), mutual_nn_rate(), reference_separability(), retrieval_margin(), uniformity_loss()

---

Decimal places similarities are rounded to before any ranking decision

Description

Equally similar references are common in real data: shared tokens, anagrams, duplicated records. Whether such a pair is *seen* as tied otherwise depends on the last bits of a dot product, which differ between BLAS builds and languages. Rounding first makes "equally similar" mean the same thing everywhere, so the tie-break by ascending index gives the same answer. Reported scores are left unrounded.

Usage

RANK_DECIMALS

---

Reference-set separability

Description

How well distinct references are kept apart. Returns the mean and median cosine to the nearest *other* reference (lower is better), and the fraction whose nearest other reference exceeds 'confusion_threshold'.

Usage

reference_separability(ref_emb, confusion_threshold = 0.9)

Arguments

ref_emb	Numeric matrix of reference embeddings.
confusion_threshold	Cosine above which two references count as confusable.

Details

These are raw cosines, so they partly reflect anisotropy: transformer embeddings occupy a narrow cone, pushing every cosine high. Retrieval depends on the ordering of similarities, not their level, so [centered_separability()] gives the anisotropy-corrected view.

Value

Named list of 'mean_nn_similarity', 'median_nn_similarity', 'confusability_rate'.

See Also

Other assess metrics: alignment_loss(), centered_separability(), hubness(), intrinsic_dimensionality(), mutual_nn_rate(), positive_pair_rank(), retrieval_margin(), uniformity_loss()

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
refs <- bag(c("aspirin", "ibuprofen", "paracetamol", "metformin"))
queries <- bag(c("asprin", "ibuprofn", "paracetmol", "metfornin"))
reference_separability(refs)

---

Retrieval margin

Description

How decisively the top reference beats the runner-up for each query, both in raw cosine and in per-query standard deviations ('_z', which is scale-free and so comparable across models with different cosine ranges).

Usage

retrieval_margin(query_emb, ref_emb, low_margin = 0.02)

Arguments

query_emb, ref_emb	Numeric embedding matrices.
low_margin	Margin below which a match counts as ambiguous.

Value

Named list of 'mean_margin', 'low_margin_rate', 'mean_margin_z', 'low_margin_z_rate'.

See Also

Other assess metrics: alignment_loss(), centered_separability(), hubness(), intrinsic_dimensionality(), mutual_nn_rate(), positive_pair_rank(), reference_separability(), uniformity_loss()

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
refs <- bag(c("aspirin", "ibuprofen", "paracetamol", "metformin"))
queries <- bag(c("asprin", "ibuprofn", "paracetmol", "metfornin"))
retrieval_margin(queries, refs)

---

Register a backend that turns model names into embedding functions

Description

The package ships no model runtime of its own. Point this at whatever you use to produce embeddings, e.g. a local ONNX session, an API client, or a cached matrix.

Usage

set_embedding_backend(backend)

get_embedding_backend()

Arguments

backend

A function taking a model name and returning a function 'f(texts) -> matrix'. 'NULL' clears the registration.

Value

The previous backend, invisibly.

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
old <- set_embedding_backend(function(model_name) bag)
as_embedder("any-model-name")
set_embedding_backend(old)

---

Normalise a string by sorting its whitespace-separated tokens

Description

The preprocessing step inside [token_sort_ratio()], exported because the same normalisation is useful when caching or de-duplicating inputs.

Usage

token_sort(x)

Arguments

x	Character vector.

Value

Character vector with tokens sorted and single-space separated.

Examples

token_sort("  South 24 Parganas ")

---

Token-sort similarity ratio

Description

Reproduces 'rapidfuzz.fuzz.token_sort_ratio', the scorer behind the default '"rapidfuzz"' backend.

Usage

token_sort_ratio(a, b)

Arguments

a, b	Character vectors, recycled against each other.

Details

Exact agreement rests on an identity: RapidFuzz's 'ratio' is the normalised indel (insert/delete-only edit) similarity,

ratio(a, b) = 100 * (1 - indel(a, b) / (nchar(a) + nchar(b)))

and 'stringdist(method = "lcs")' computes that indel distance. Both sides evaluate the same rational number in double precision, so the scores agree to within an ULP.

Tokens are split on whitespace, sorted, and rejoined with a single space. Case and punctuation are left alone, so '"bombay"' and '"Bombay"' score 83.33, not 100.

Value

Numeric vector of scores in '[0, 100]'.

Examples

# Token order does not matter, but case does.
token_sort_ratio("New Delhi", "Delhi New")
token_sort_ratio("Bombay", "Mumbai")

---

Top-1 accuracy and MRR of an embedder against ground truth

Description

The labelled counterpart to the label-free metrics: given the correct reference for each query, how often does the model retrieve it? [assess_models()] predicts this without labels.

Usage

true_accuracy(queries, references, truth, model, nil_threshold = NULL)

Arguments

queries	Character vector of query strings.
references	Character vector of candidate strings.
truth	Correct reference per query, or 'NA' for an open-world NIL query with no correct answer. Real matching input is full of these: a cause of death that is not a diagnosis, a drug outside the formulary.
model	Anything [as_embedder()] accepts.
nil_threshold	Cosine below which a top-1 match counts as an abstention. Required when 'truth' contains 'NA'; ignored otherwise.

Value

A named list with 'top1' and 'mrr', plus 'nil_recall' and 'answerable_top1' when the dataset contains NIL queries. Those two are separate because one accuracy number hides the trade-off: a model can look good by always abstaining, or never.

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
true_accuracy(
  c("colour", "flavour"), c("color", "flavor"),
  c("color", "flavor"), bag
)

---

Uniformity loss

Description

'log E exp(-t * ||x - y||^2)' over all pairs. More negative means the embeddings spread more evenly over the sphere, wasting less of the representation. After Wang & Isola (2020).

Usage

uniformity_loss(emb, t = 2, max_points = 2000)

Arguments

emb	Numeric matrix of embeddings.
t	Temperature.
max_points	Subsample size when there are more rows than this.

Value

Scalar loss.

See Also

Other assess metrics: alignment_loss(), centered_separability(), hubness(), intrinsic_dimensionality(), mutual_nn_rate(), positive_pair_rank(), reference_separability(), retrieval_margin()

Examples

bag <- function(x) {
  t(sapply(
    strsplit(tolower(x), ""),
    function(ch) tabulate(match(ch, letters), 26)
  ))
}
refs <- bag(c("aspirin", "ibuprofen", "paracetamol", "metformin"))
queries <- bag(c("asprin", "ibuprofn", "paracetmol", "metfornin"))
uniformity_loss(refs)

---

Resolve bare model names through the local ONNX runtime

Description

After calling this, a model name passed to [alethia()], [cluster_entities()] or [as_embedder()] is loaded and run locally on the CPU.

Usage

use_onnx_backend(...)

Arguments

...	Passed to [onnx_embedder()], e.g. 'threads' or 'batch_size'.

Value

The previously registered backend, invisibly.

Examples

if (onnx_available()) {
  use_onnx_backend(threads = 4)
  alethia(c("Bombay"), c("Mumbai", "Delhi"), model = "sentence-transformers/all-MiniLM-L6-v2")
}

---