Releases | Open Source Science

teiphy - New mean-idf, mi, and mean-mi --table options

This release adds three new tabular formats accessible with the --table option. The first is mean-idf, which calculates the mean inverse document frequency (IDF) of two witnesses over all variation units where both have non-zero reading support vectors (i.e., where neither is lacunose, assuming no --split-missing option has been specified), including variation units where those witnesses disagree. The second is mi, which calculates the total mutual information (MI) of two witnesses over all variation units where both have non-zero reading support vectors. At each such variation unit, this corresponds to the Kullback-Leibler divergence of the observed joint distribution of the witnesses' readings from the expected joint distribution of their readings under the assumption that the witnesses are independent. The third format is mean-mi, which calculates the average MI rather than the total MI. The mean-idf and mean-mi formats will improve the value of relationships involving more fragmentary witnesses, but if extremely fragmentary witnesses are not filtered out the with --fragmentary-threshold, then their values may be overinflated.

Scientific Software - Peer-reviewed - Python
Published by jjmccollum 11 months ago

teiphy - Multiple --split-missing options and support for IDF-weighted agreement tables

This release allows multiple options for the --split-missing option for tabular inputs. This option was previously a boolean flag. The uniform option replicates its old behavior: where a witness has missing data in a variation unit, its values for each of its potential readings in that unit are assigned equal proportions of the value 1. This corresponds to a uniform prior on which reading it originally had. The proportional option corresponds to a prior informed by the sample populations of the variant readings among the witnesses that are not missing data in that variation unit.

In conjunction with this new feature, this release also supports a new idf argument for the --table option. With this argument, teiphy produces an agreement matrix with rows and columns for witnesses where the agreements are weighed by their inverse document frequency (IDF). Equivalently, the cell for two witnesses contains the total expected information content (in bits) of their agreements at all variation units, where the probability of agreement on a reading is defined as the sampling probability of that reading. Effectively, this weighting scheme treats more exclusive agreements as more informative. In this way, it can help us get behind certain layers of textual influence, such as assimilation to a popular text, to highlight relationships that general agreement counts or proportions cannot. If no --split-missing argument is specified, then variation units where either witness has missing data are ignored. If either --split-missing argument is provided, then the expected information content of an agreement is determined based on the corresponding priors about which reading one or both witnesses might have had. In practice, ignoring variation units where one or both witnesses are missing data results in fewer false positives with fragmentary witnesses.

Scientific Software - Peer-reviewed - Python
Published by jjmccollum 11 months ago

teiphy - Support for weighted variation units in stemma, updated example file, updated BEAST 2 template

The biggest contribution of this release is the addition of support for weighted variation units (specifically when generating inputs for the stemma program). Another significant contribution is a sweeping update to the example file to reflect the UBS collation of Ephesians more closely. Some unexpected behavior in the validation of witness dates was addressed, so that the check for whether the latest date is fixed in at least one witness now correctly gives preference to witnesses that have precise dates instead of date ranges. Finally, the template for BEAST 2.7 XML outputs now includes a tree initialization element and updated parameters and priors for the birth-death-sampling tree prior (Stadler, et al. 2013) to account more precisely for the use of dated tips (common for stemmata).

Scientific Software - Peer-reviewed - Python
Published by jjmccollum about 1 year ago

teiphy - Support through Python 3.12, 62-state support for NEXUS outputs, and support for PHYLIP distance/similarity matrices

This release incorporates contributions from @catsmith to ensure compatibility with Python versions 3.9 through 3.12. (As a result of these changes, Python 3.8 is no longer supported.) To accommodate software like PAUP* and @edmondac's fork of MrBayes (https://github.com/edmondac/MrBayes), the symbol set for NEXUS outputs has been extended to 62 symbols (0-9, a-z, and A-Z). This release also adds support for the use of --table distance and --table similarity options (along with the --proportion and --show-ext flags) with outputs in PHYLIP (.phy and .ph) format to produce PHYLIP-formatted distance or similarity matrices.