Quickstart

Install

# install.packages("remotes")
remotes::install_github("saketlab/alethia", subdir = "R")

Match against a reference list

The default matches on spelling and needs no model.

messy <- c("New Yrok", "Los Angelos", "Chicagoo", "Houston")
reference <- c("New York", "Los Angeles", "Chicago", "Houston", "Phoenix")

alethia(messy, reference)
#>   given_entity alethia_prediction alethia_score alethia_method
#> 1     New Yrok           New York     0.8750000      rapidfuzz
#> 2  Los Angelos        Los Angeles     0.9090909      rapidfuzz
#> 3     Chicagoo            Chicago     0.9333333      rapidfuzz
#> 4      Houston            Houston     1.0000000          exact

Match on meaning

as_embedder() takes any function returning one row per string, so no download is needed to see the embedding path work. A character-count embedder stands in for a real model here.

char_bag <- function(x) {
  t(vapply(strsplit(tolower(x), ""), function(ch) {
    tabulate(match(ch, c(letters, " ")), 27)
  }, numeric(27)))
}

match_by_embeddings(messy, reference, char_bag)
#>   given_entity alethia_prediction alethia_score
#> 1     New Yrok           New York     1.0000000
#> 2  Los Angelos        Los Angeles     0.9411765
#> 3     Chicagoo            Chicago     0.9622504
#> 4      Houston            Houston     1.0000000

Pass a model name instead once an ONNX backend is registered; see ?set_embedding_backend.

Choose a model without labels

assess_models() scores candidates on your own data. The composite is relative, so it needs at least two.

reversed <- function(x) {
  char_bag(vapply(strsplit(x, ""), function(ch) paste(rev(ch), collapse = ""), ""))
}

res <- assess_models(messy, reference, list(char_bag = char_bag, reversed = reversed))
res$table[, c("model", "score", "mean_margin_z", "mutual_nn_rate")]
#>      model score mean_margin_z mutual_nn_rate
#> 1 char_bag     0      1.865052              1
#> 2 reversed     0      1.865052              1
res$best
#> [1] "char_bag"

Cluster without a reference list

Edges need mutual nearest-neighbour agreement, and each carries a confidence.

entities <- c("Bombay", "Bombay City", "Calcutta", "Calcutta City", "Madras")
clusters <- cluster_entities(entities, char_bag, floor = 0.9)

clusters$clusters
#> NULL
clusters$edges
#>   i j    cosine  margin_i  margin_j mutual confidence
#> 1 3 4 0.9258201 0.5117808 0.3295353   TRUE   1.230911