Recent Releases of g2p

:bug: Bug Fixes

82d4292 - minimal fixes to get Dockerfile working again (PR #427 by @joanise)
3b465cd - fix 426 by limiting the number times we escape special characters (commit by @roedoejet)

:whitecheckmark: Tests

e7703b2 - add a test case exercising escape-special (commit by @joanise)

- Python
Published by github-actions[bot] 9 months ago

:sparkles: New Features

9dbd466 - add Halkomelem APA/Hul’q’umi’num’ mapping (commit by @goodzack)
ed987b3 - chain equiv for kwk-umista; handle case better for all kwk mappings (commit by @joanise)
403b0a3 - kwk broad IPA mapping for a more phonemic IPA representation (commit by @joanise)

:bug: Bug Fixes

f1dac49 - name ikt "Inuktut, Western" so it sorts next to Inuktitut (PR #410 by @joanise)
bc7e599 - app.py assumes a Transducer, just type ignore the mypy error (commit by @joanise)
121cb92 - test: make test_studio more robust by waiting for meaningful effects (commit by @joanise)
65305d4 - kwk digraphs in Umista use U+0315 not U+0313 (commit by @joanise)
6b03886 - kwk-umista-equiv had a typo for y\u0313 -> 'y (commit by @joanise)
fea19a1 - regenerate kwk-ipa -> eng-ipa (commit by @joanise)
5a5e570 - adjust kwk mapping in collaboration with Daisy R (commit by @joanise)
8f1abe7 - kwk: regen kwk-ipa -> eng-ipa (commit by @joanise)
dac2976 - revert kwk umista->ipa to stricter IPA (commit by @joanise)
e06fe72 - call it phonemic IPA, not broad, by AP-PL-EJ consensus (commit by @joanise)
aca5701 - hur->hur-apa is from orthog to APA, not vice versa (PR #423 by @joanise)

:recycle: Refactors

0be84f5 - change 'hur_orthog' name to 'hur' (commit by @roedoejet)
2b77e4e - test: name the 4 test case values instead of using indices (commit by @joanise)
2ac44b6 - remove explicit indices from kwk when they are monotonic (commit by @joanise)
9d4cdd4 - rename kwk-con mappings kwk-equiv (commit by @joanise)
a98ef00 - explicitly declare that kwk mappings rely on NFD norm_form (commit by @joanise)
eafdde0 - chain kwk-broad-ipa from kwk-ipa to make it DRY (commit by @joanise)

:whitecheckmark: Tests

8b67292 - make sure the SCMPRETENDVERSION is up to date before publishing (commit by @joanise)
085639a - refactor test_studio and make it more efficient (PR #420 by @deltork)
2af6a73 - refactor teststudio to use expect.tohaveattribute for outlang (commit by @joanise)
57a959a - kwk: some test data for Umista text (commit by @joanise)
4d33f9e - update tests for broad and strict kwk IPA (commit by @joanise)

:wrench: Chores

5f0adf9 - bump version in .SETUPTOOLSSCMPRETEND_VERSION (commit by @joanise)
09a0590 - remove obsolete network.pkl file (commit by @joanise)
8e38a52 - update pre commit configuration (commit by @joanise)
09736d1 - g2p update (commit by @joanise)
a476bfa - g2p update to roll in latest kwk changes (commit by @joanise)

- Python
Published by github-actions[bot] 11 months ago

:sparkles: New Features

1262cbb - add --quiet option to tests/run.py and refactor the runners (commit by @joanise)
c419518 - add a lexicon-based tokenizer, esp. for English (commit by @joanise)

:bug: Bug Fixes

419507e - indent only the first line in click indented paragraphs (commit by @joanise)

:zap: Performance Improvements

24a28e0 - prevent quadratic time cost of degenerate inputs for lexicon-based tok (commit by @joanise)

:recycle: Refactors

cf38989 - tests: quiet and reformat some test suites (commit by @joanise)
5682125 - simplify mergeifsamelabel to clearer mergesametypetokens (commit by @joanise)
d662622 - move mergenonwordtokens and splitnonwordtokens to utils (commit by @joanise)
163bc39 - import utils as a whole instead of each function (commit by @joanise)
c3d73bf - change tokens from a a custom dict to a Token class (PR #406 by @joanise)

:whitecheckmark: Tests

2b8a803 - heroku: exercise the real Heroku server command in CI (commit by @joanise)
0b2c83c - better unit testing for mappings.utils (commit by @joanise)

:wrench: Chores

b2bd476 - migrate the pre-commit config to 4.x style (commit by @joanise)

- Python
Published by github-actions[bot] over 1 year ago

This is a primarily a performance improvement patch, reducing the memory footprint by about 45MB, and the initial load time, by: - using a more compact in-memory structure for the English lexicon, and - replacing the heavy-weight networkx library by a tiny custom class implementing only the algorithms used.

:sparkles: New Features

966a057 - allow panphon 0.21 where possible (commit by @joanise)
aa9de1c - g2p show-mappings to display language names too (commit by @joanise)
c70f30f - network_lite with minimal DiGraph class (commit by @joanise)
123e27b - add full type signatures to DiGraph (commit by @dhdaines)
6eb29ac - revamp schema versioning and update-schema (commit by @joanise)

:bug: Bug Fixes

8929608 - add [tool.setuptools_scm] in pyproject.toml to please the build system (commit by @joanise)
208a8e0 - deps: pydantic 2.9 changes our schemas, so block it (commit by @joanise)
16668b2 - enable type-checking and fix things (commit by @dhdaines)
6ab8545 - make sure self.rules is always the type we say it is (commit by @dhdaines)
3eee1a6 - seeing matchpattern or intermediateform is an error (commit by @dhdaines)
bbcd1e8 - avoid unnecessarily requiring a schema update (commit by @joanise)

:zap: Performance Improvements

e605ae5 - compact lexicon entries to take less RAM (commit by @joanise)
96abff3 - replace networkx by network_lite throughout reduces memory footprint and load time (commit by @joanise)

:recycle: Refactors

d1b3437 - simplify shortest_path code (commit by @dhdaines)
e2def43 - only declare the SCM pretend version in one place (commit by @joanise)

:whitecheckmark: Tests

d03aabb - carefully cover compact lexicon corner cases (commit by @joanise)

:wrench: Chores

0523ba9 - g2p update (commit by @dhdaines)
72e761d - g2p update, to get compacted lexicon (commit by @joanise)

- Python
Published by github-actions[bot] over 1 year ago

g2p - v2.1.0

:boom: BREAKING CHANGES

due to 74e6172 - reimplement v1 API with FastAPI (commit by @dhdaines):

/api/v1 error status code for validation errors is always 422, no longer 400 or 404

:sparkles: Major New Features

74e6172 - reimplement v1 API with FastAPI (commit by @dhdaines)
605ccd3 - reimplement Studio app with FastAPI (commit by @dhdaines)
c214c6f - add /api/v2 to studio but also make it standaloneable (commit by @dhdaines)

:sparkles: New Features

36e4dcc - switch to hatch and dynamic versioning (commit by @dhdaines)
e0a0219 - build: autogenerate requirements.txt with hatch-pip-compile (commit by @dhdaines)
1fe3385 - add a G2P_LOGLEVEL environment variable (commit by @dhdaines)
bd33314 - add redirections for backward compatibility (commit by @dhdaines)
74c5c47 - new API supporting textual alignments (commit by @dhdaines)
7909e6e - Add sal-apa generic mapping for APA-based Salish writing systems (commit by @joanise)
077afc2 - add logic to auto-delete as_is support in g2p 3 (commit by @joanise)
d4bffad - g2p convert accepts - for stdin and linux /dev/ pipes (commit by @joanise)
f0cf073 - g2p convert now accepts --file option to read a file (commit by @joanise)
a938917 - bump the current major.minor version to 2.1 (commit by @joanise)

:bug: Bug Fixes

1cc2afe - ci: eventlet 0.36.0 considered harmful (commit by @dhdaines)
d6004f9 - style: bump black to 24.3.0 to fix black's first CVE (commit by @joanise)
05f51f9 - do not try to send whole lexicon over the wire (commit by @dhdaines)
49ad2ff - port 5000 is used by MacOS on external interfaces (commit by @dhdaines)
629209b - test: use 127.0.0.1 explicitly to avoid ipv6 confusion (commit by @dhdaines)
d105e5f - allow other mapping arguments, use on-disk alignments (commit by @dhdaines)
b29b23f - ci: eventlet 0.36.0 considered harmful (commit by @dhdaines)
baef8fd - ci: remove bogus sleep (commit by @dhdaines)
52b3bfd - needed apply-longest-first for atj (since the beginning (commit by @dhdaines)
d9a07e5 - do not copy the input mapping filename when generating (commit by @dhdaines)
ea04262 - do not try to generate mappings for empty outputs (commit by @dhdaines)
f50768e - g2p convert should not add newline when input is a file (commit by @joanise)
561817c - deps: specific anti-dependency on broken coloredlogs version (commit by @dhdaines)
9f92f65 - deps: use optional dependencies correctly (for docs too) (commit by @dhdaines)
c8cba5f - test: no longer require flask needlessly for some tests (commit by @dhdaines)
1a602ca - build: various build fixes (commit by @dhdaines)
9543c96 - deps: old versions of eventlet are also broken (commit by @dhdaines)
4e6c3ab - docs: add install link for hatch (commit by @dhdaines)
656f07a - ci: ensure version matches schema (commit by @dhdaines)
4e23d76 - docs: mention conda (commit by @dhdaines)
38d5290 - build: add a hook to make sure we have g2p/_version.py on heroku (commit by @dhdaines)
1bba827 - update API for newer FastAPI (commit by @dhdaines)
5922f6f - get Studio working with FastAPI (commit by @dhdaines)
98a07f1 - restore compatible 404 response and enable api tests (commit by @dhdaines)
89bd9b3 - deps: fix deps for api (commit by @dhdaines)
cfc50c6 - update prod environment and workflow (commit by @dhdaines)
ebc16ff - now need python 3.8 on windows (commit by @dhdaines)
0a7c78b - not sure why we need to disable sendfile (commit by @dhdaines)
4bcd948 - remove fastapi-socketio (commit by @dhdaines)
d5d2086 - make the g2p library tests still run on Python 3.7 (commit by @joanise)
f55e6bb - ci: make coverage work again (commit by @dhdaines)
9bc3855 - test: fix coverage (commit by @dhdaines)
ff6c92d - more specific dependency to avoid gnashing of teeth (commit by @dhdaines)
2d68577 - deps: correct the gunicorn dependency... again (commit by @dhdaines)
5e3c0f1 - split /langs and /nodes as they are not the same thing (commit by @dhdaines)
a88df6a - build: depend on gitlint-core, not gitlint (commit by @joanise)
f126e1d - studio: studio is same-origin so no CORS, also add debug option (commit by @dhdaines)
9f88fbf - studio: make deleting entire input work right (commit by @dhdaines)
2a18cdd - ci: enable G2PSTUDIODEBUG to satisfy coverage (commit by @dhdaines)
30b572a - normalize ó in mohawk (commit by @MENGZHEGENG)
e6a1280 - app: do not rely on running at the g2p root dir (commit by @joanise)
627ca2e - tests: silence the logs in testapiresources tests (commit by @joanise)
54fc772 - deps: pin panphon to 0.19-0.20 as 0.21 breaks many things (commit by @dhdaines)
3323eb4 - ci: remove stale job dependency in pythonpublish workflow (commit by @joanise)
f0c744b - ci: remove broken fetch-tags from test workflow (commit by @joanise)
80b55a2 - ci: drop python 3.7 unsupported EOL from matrix testing (commit by @joanise)
95a773e - test: remove expired deprecation test (commit by @joanise)
6843fd8 - ci: update stale actions in python publish script (commit by @joanise)

:zap: Performance Improvements

b772bd6 - make g2p convert line-oriented when processing a file (commit by @joanise)
e6c006d - read files as you process them, not up front (commit by @joanise)

:recycle: Refactors

13b2f81 - simplify and clarify the failure output of test_langs.py (commit by @joanise)
5351133 - rename types->shared_types to avoid collision (commit by @joanise)

:whitecheckmark: Tests

55a8694 - studio: make sure we test one lexicon mapping (commit by @joanise)
b971231 - add coverage for xsampa mapping (commit by @dhdaines)
0e4dc75 - fix tests (and put api2 in a separate env) (commit by @dhdaines)
a0aa8c5 - exercise g2p-studio switching output langs (commit by @joanise)
fc43dbd - moh: add moh tests for ó normalization (commit by @roedoejet)
3405626 - add a (near) pep440 unit test (commit by @joanise)

:wrench: Chores

9826ceb - g2p update (commit by @dhdaines)
ff7de51 - add ignores for untyped modules (commit by @dhdaines)
24ada71 - deps: remove dependencies that were never used directly (commit by @dhdaines)
fdacd74 - delete deprecated code (commit by @joanise)
f4683a9 - deps: update pyproject and requirements (commit by @dhdaines)
a6aeb2f - g2p update (commit by @joanise)
8dafbeb - run g2p update (commit by @roedoejet)

- Python
Published by github-actions[bot] over 1 year ago

g2p - v2.0.0

Please visit the migration guide for information on how to update 1.x mappings to g2p 2.x!

:boom: BREAKING CHANGES

due to 1d8e4fb - switch to pydantic 2 (commit by @roedoejet):

requires python 3.7

:sparkles: New Features

fd33a26 - cli: add update-schema command (commit by @roedoejet)
f85c4f2 - use json for network as well (commit by @dhdaines)
b01ec23 - upgrade networkx now that we can (commit by @dhdaines)
9fe200d - schema: update schema generation to include dialect spec by default (commit by @roedoejet)
a04aeff - add case preservation option to mappings (commit by @roedoejet)
c31c66b - g2p-studio also needs to support preserve_case (commit by @joanise)
7447fe6 - make x caron equiv to x dot below in clm (commit by @joanise)
d4fdc8c - str: accept space+comb-cedilla or space+comb-comma as equiv to cedilla (commit by @joanise)

:bug: Bug Fixes

20e3bcb - pkl: remove generated default date (commit by @roedoejet)
22644e7 - studio: refactor to 'rules' instead of 'mapping' key (commit by @roedoejet)
30dc282 - ci: require 3.8 for windows ci (commit by @roedoejet)
1df2dfd - add miscellaneous style fixes and typos (commit by @roedoejet)
5ccd595 - update: prevent loading all the mappings multiple times (commit by @roedoejet)
45d5ecf - tests: fix studio tests (commit by @roedoejet)
16e4869 - restore Python 3.7 compatibility (commit by @joanise)
060a8aa - use more generic variable names (commit by @dhdaines)
ac2d42d - deps: back off networkx dep for python 3.7 (commit by @dhdaines)
fa27730 - crg: fix various rule feeding and ordering bugs for Michif (commit by @joanise)
007aef5 - crg: manually clean up crg-ipa -> eng-ipa (commit by @joanise)
0e9271a - test: fix failure in test failure (commit by @dhdaines)
15d5b64 - test file could have arbitrary extra fields (commit by @dhdaines)
25f4713 - output a compatible config-g2p.yaml though some filenames change (commit by @dhdaines)
32fe87c - add configonly option to exportto_dict (commit by @dhdaines)
b5f9747 - um, yes, model_dump() exists (commit by @dhdaines)
2e5e560 - do not exclude defaults, just inappropriate keys for config (commit by @dhdaines)
9975100 - add missing double vowel vowels to crg (commit by @dhdaines)
83b6c1c - cursèd unicode g strikes again (commit by @dhdaines)
f766a66 - remove werkzeug lock since it is no longer necessary (commit by @joanise)
1c7792f - correct the unit testing output for g2p mapping errors (commit by @joanise)
996a060 - remove unused kwargs in transducer call (commit by @roedoejet)
d1aa6dd - sort rules without explicit indices (commit by @roedoejet)
d768d74 - detect incompatible casesensitive+preservecase instances (commit by @joanise)
35868bb - preserve indices through prevent-feeding intermediate form (commit by @joanise)
01ff75e - fix coverage issues and grepping for slow imports (commit by @joanise)
251739a - deps: lock numpy<2 because 2.0.0 is coming and has breaking changes (commit by @joanise)
27d0d2d - rename crj and crl "East Cree, Nor/Southern" so they sort nicely (commit by @joanise)
17519d8 - y in oka should go to /j/, palatal glide, not /y/ (commit by @joanise)
b52a819 - issue a fatal error when reading an empty mapping (commit by @joanise)
95bf4be - app: errors in mappings should just trigger console warnings (commit by @joanise)
5993242 - str: cedilla is now the default glottal stop character (commit by @joanise)
d18d17a - publish schemas only for major.minor, ignoring .patch (commit by @joanise)
f2a7563 - assertEquals is removed from Python 3.12 (commit by @joanise)
5592659 - close xlsx workbook after reading (commit by @joanise)
7f34057 - loading xlsx workbooks should not fail on empty cells (commit by @joanise)

:zap: Performance Improvements

a5f51b7 - only create APP when it is really needed (commit by @joanise)
0b8d773 - defer a whole bunch of expensive imports from the CLI (commit by @joanise)
978153b - remove the app from the cli to make the CLI faster (commit by @joanise)

:recycle: Refactors

eec8e82 - massive refactor to pydantic (commit by @roedoejet)
1d8e4fb - switch to pydantic 2 (commit by @roedoejet)
a753e07 - config: require a 'mappings' key (commit by @roedoejet)
006d370 - inchar and outchar to ruleinput and ruleoutput (commit by @roedoejet)
b448523 - change to config-g2p.yaml (commit by @roedoejet)
5a67040 - change langs.pkl to langs.json (commit by @roedoejet)
5b259ff - separate data and path for rules, abbreviations, and alignments (commit by @roedoejet)
ddefe77 - make mapping.rules the only way to get to the rules (commit by @joanise)
090145e - docs: change docs to mkdocs (commit by @roedoejet)
3c8f51a - docs: add nav footer and move python example (commit by @roedoejet)
df0b61e - move stripindexnotation function to general utils module (commit by @roedoejet)
8363cfa - move stripindexnotation to mappings/utils.py (commit by @joanise)
5d2e3bb - use ABCs to keep typing despite deferring imports (commit by @joanise)
c9c7c1e - put types in types.py for more clarity (commit by @joanise)
2379fe4 - remove dead code, create_app was only needed for the cli (commit by @joanise)
b79480f - replace deprecated pydantic v1 @validator by v2 equivalents (commit by @joanise)
91e03a0 - make test runners DRYer and let dev be default (commit by @joanise)
414c812 - version numbers are now PEP 440 compliant, not date based (commit by @joanise)

:whitecheckmark: Tests

482fbc6 - add a test case for generate-mapping --ipa (commit by @joanise)
3811a9e - kwk: minimally exercice kwk BOAS->Umist case preservation (commit by @joanise)
4c3e102 - exercise index preservation through prevent-feeding (commit by @joanise)
2156be8 - exercise create_app() in unit testing (commit by @joanise)

- Python
Published by github-actions[bot] almost 2 years ago

g2p - Release v1.1.20230822

1.1.20230822 (2023-08-22)

Features

deps: make dependencies dependant on the Python version (6e68140)
clm (Klallam) mapping to g2p (882925a)
moh: update moh mappings (14e8bc6)

Bug Fixes

bisect_left does not accept key before Python 3.10 (cbb9fb2)
updating flask means updating socketio means updating socket.io.js (785f668)
deps: make sure engineio and socketio are all compatible (600b2ec)
have generate-mapping create files that pass pre-commit hooks (f6494a9)
the egg syntax is deprecated, use the at syntax instead (697abcb)
deps: lock dnspython to compatible 2.3.0 (e4eaa96)
^ and $ are null-length so require separate sorting for creating fixed-width lookbehind (1ef573b)
error with missing apostrophe (8e55e44)
mapping: fix bug in haa mapping and add test suite lookbehind construction (a9e5e69)
moh: change name of language to Kanien'kéha (e3ab8c3)
studio: pin hands on table to 12.4 (b7df593)

Performance Improvements

build only in_seq or mappings as needed for alignments (4e6de3b)
store lexicon alignments as strings to save memory (6543214)
store lexicon k:v entries as joined strings, even less RAM (b984c42)

Tests

add unit test case mimicking #130 to confirm it works on Windows (b413089)
exercise the short -h option in unit testing (40db7fc)

Build Systems

bump gunicorn to latest version, just published (01234c7)
bump Heroku runtime to 3.10.12 as per Heroku warning (7f249d9)
force Heroku to bump python to 3.10.11, and docs (a0b9c03)

Continuous Integration

only run the full matrix test on release (f02f1ff)
reorganize CI test suites (c04c660)
run matrix tests on push to main too since that gets deployed (2622913)

Documentation

tell the user they need python 3.7 if they try to run studio with older (50852d8)
update phoneset (5eb14b1)

Code Refactoring

apply dhd feedback to remove dead code and unflatten the alignment (324e1a2)

- Python
Published by github-actions[bot] over 2 years ago

g2p - Release v1.1.20230511

1.1.20230511 (2023-05-11)

⚠ BREAKING CHANGES

makeg2p(in, out) used to not tokenize, now it does, and its toklang argument is deprecated
g2p convert now tokenizes by default

Features

expose the tokenize option to api/v1/g2p (3f572c4)
g2p convert now tokenizes by default (4d67902)
make_g2p now tokenizes by default and has new signature (ecfe2ca)

Bug Fixes

adjust all calls to make_g2p to its new signature (bea7cec)
g2p needs to update both generated .pkl and .json files (2be51f8), closes #237
remove --path option to g2p convert, which does not work anyway (f99774f)
use the more canonical DeprecationWarning to flag deprecation (e8a8a4d)
mappings: output should not be escaped (5bd3250)

Documentation

add tokenize arg for api/v1/g2p to swagger.json (d2f226f)

Continuous Integration

make test_studio.py fast enough to run on each push (5fa2a01)
remove unused coveralls, make our omit compat with coverage 7.x (3f9d2df)

Tests

execise api/v1/g2p with and without tokenize (c64322f)
improve coverage of error situations in CLI (0b3f5ee)

Code Refactoring

make Tokenizer the base class name, and declare to return types (7c8e8f1)
move deprecation and version checking code to their own file (e61daa4)
remove dead code in app.py, increase test cov and speed up tests (07e87d6)

- Python
Published by github-actions[bot] almost 3 years ago

g2p - Release v1.0.20230417

1.0.20230417 (2023-04-17)

Bug Fixes

eng is already in the langs now, no need to hardcode (b038ad2)
import g2p should not alter sys.stdout/err globally (80e0d1b)
the CLI (and only the CLI) needs to ensure utf8 output on Windows (cbeff1f)

Code Refactoring

move get_langs from Studio/readalongs to g2p (c06ae5d)
rename getlangs->getarpabet_langs to make purpose clearer (7c5222e)

Continuous Integration

annotate version tags (99b1747)
make sure the CLI outputs utf8 on Windows (2612a1a)
tell codecov to ignore the utf8 patch for Windows (4339009)

- Python
Published by github-actions[bot] almost 3 years ago

g2p - Release v1.0.20230412

1.0.20230412 (2023-04-12)

⚠ BREAKING CHANGES

put networktoechart where we can test it properly

Features

add -a/--substring-alignments argument to cli (6b41213)
add accessors for useful things like the input and output languages (cacce3b)
add aligned cmudict and lexicon transducer type (596ab82)
add alignments method to get textual alignments (e2303f4)
add edges for alignments in lexicon (f2c9f6c)
add proper typing to compose_indices (7bbfb6d)
add type checking and use Tuples (as they can be type checked) (4780702)
language name for spelling variants describe the variant (ffba389)
make the use of None explicit and limited (97aaed5)
make TransductionGraph and CompositeTransductionGraph compatible (e00790a)
output monotonic alignments for deletions and reorderings (126aa83)
properly normalize edges on concatenation (f37897c)
shrink pickle by optimizing alignment storage (0860ad6)
support lexicon mappings in Studio (but they are slow) (c824f6b)
switch script to use phonetisaurus from PyPI (bb91b12)

Bug Fixes

add spaces and avoid formatting (a5c2894)
avoid crashing on empty edges (8d57e68)
avoid creating None in input position (404306d)
comment and clean up substring_alignments (9cd84d8)
disable the utf8 fix for windows when running in pytest (bd5690a), closes #241
do not call logging.basicConfig, just config the logger itself (8ff314f)
emit input unchanged when no transducers exist (b0db10e)
fix doctor (0b0f2ed)
fix speed issues by not deep-copying alignments (56e933b)
make pretty_edges consistent and fix tests to expect tuples (065fa23)
make sure we do not output bogus edges (fab9f0a)
most sensible possible behaviour, keep spaces if user wanted them (70ab1e6)
remove impossible try/catch (2db239a)
remove spaces in sanitize_unidecode_output as suggested by @littell (bd1b1ec)
remove spontaneous extraneous spaces from und-ipa (9e64b7f)
remove unnecessary default value (722215a)
restore original edges API and rename alignments (c054256)
switching back to Custom did not actually work (7f0f640)
the only special character we want to escape is ? (7af2f0b)
update treatment of deletions in lexicon to match rules (18bdc6b)
use OrderedDict explicitly for clarity (d2ef567)

Documentation

add documentation for lexicon mappings (dcf5973)
add links to non-packaged files (9d6275c)
clarify use of generic type (7bb7df6)
clean up docstrings (91aa3b3)

Tests

add alignment tests and improve coverage for tranducers (76f85dd)
add coverage of invalid regex in rule (bd81a70)
add coverage to studio tests and app (0945336)
add test of lexicon loading from config file (22de19b)
fix studio test (31c9e48)
long delay no longer necessary (33efc1e)
make test_tokenizer.py exercise tce and unknown lang and default (1da815b)
run the expensive doctor test because it can catch errors (bb60f55)
update lexicon test for eng ipa (f05a513)

Code Refactoring

add explicit b, m, p, u rules to moh for borrowed words (2dc5e42)
put networktoechart where we can test it properly (970e358)
remove superfluous list comprehension (dd8f5df)
test: when a mapping fails, show test case filename:lineno (fb309ec)
tests: quiet yappy test suites (c6423b6)

Styles

all other badges are rounded, why not the readme one? (ba76f57)
rewrite mohequiv and mohto_ipa in compact form (c781cbe)

Continuous Integration

replace deprecated actions/create-release by ncipollo/release-action (43d1060), closes #200
replace deprecated set-output and bump github-tag-action (8b40a1b), closes #200

- Python
Published by github-actions[bot] almost 3 years ago

g2p - Release v1.0.20230228

1.0.20230228 (2023-02-28)

Bug fixes

Fix to Han/Kanji characters in Und (6e248a08de)

- Python
Published by github-actions[bot] about 3 years ago

g2p - Release v1.0.20230224

1.0.20230224 (2023-02-24)

Features

add nsy mapping for the nsyilxcən Language (8d7f04c)
improve the g2p-studio static page (79d4257)

Bug Fixes

mappings: bullet operator -> middle dot (f5b3d06)
studio: upgrade heroku stack and python runtime (b10abee)
address CWE-830 by adding integrity to scripts from cloudflare (00ccd31)
generate swagger.json the way our pre-commit hooks want it (0373abe)
in 2022, "python" is Python 3 (38c41da)
in 2022, "python" is Python 3 and "pip" works in CI (f2b892a)
on Windows, make generated files out LF so they're not spuriously changed (0613906)
ci: add codecov token to ci tests (2fbfe06)
ci: change ubuntu version (48158a2)
nsy: add glotal stop self-map so g2p knows it is an nsy letter (05a9c75)
nsy: fix the nsy->nsy-ipa mapping to the picky requirements of g2p (b6d5389)
nsy: handle a few more spelling variants (2a51896)
make Undetermined (und) process Arabic characters correctly (53dded4)
reqs: update flask to avoid werkzeug error (361c936)

Performance Improvements

collapse loops (809adaa)

Code Refactoring

change Nsyilxcən code to oka in all the files too (b5b3a60)
change to main (1ad9a98)
create class mg-bot for cleaner bottom margin implementation (cb4ab03)
rename nsy->oka to the official iso-6639-1 code for Nsyilxcən (f0b5bd6)
docs: use unpkg for fetching swagger ui (297b069)

Styles

apply a number of pylint recommendations (2e9b067)
let git blame ignore black and isort only commits (56896de)

Tests

add --describe option to run.py and exit 1 on error (6b560f0)
test that eng-ipa->eng-arpabet works ok with NFC and NFD inputs (2712b32)
use NFD output in fn_unicode test cases (a51bb46)
nsy: add references to most entries in nsy.csv (e7f7726)
nsy: fix the last word (question mark -> glottal stop) (946d7c3)

Continuous Integration

add CodeQL automated vulnerability scanning (9fc96fd)
bump CI actions to current to heed GitHub warnings (4858bbb)
g2p codecov action does not use dir (8880845)
only run CodeQL on cron and push to master and release (11aaff4)
stop failing CI when codecov fails to upload (557fc3d)
use ubuntu-20.04 since ubuntu-latest no longer supports Python 3.6 (cb91794)

- Python
Published by github-actions[bot] about 3 years ago

g2p - Release v0.5.20221013

0.5.20221013 (2022-10-13)

Features

add dummy mappings for english and mohawk (eaf2c70)
add dummy mappings for english and mohawk (85d06f0)
add iku-ipa to hamming-eng-ipa mapping (9ae5484)
add und and str dummy mappings as well as distance specification for mapping alignment (7f93447)
mappings: added more dummy mappings (d89f5b7)
add und-ipa to hamming-eng-ipa (7cf4118)
basic Finnish mapping (36d17e0)
check Python version (9e086e1)
do arpabet checking for hamming-eng-arpabet too (11728ec)
include NFC/D normalization in g2p graph (f3b918c), closes #158 #158
und now maps colon to an empty string (0b8c8d9)
mappings: allow abbreviations to be declared recursively (e1a270f)
show-mappings: add --csv option (43573ac)
show-mappings: added cli cmd g2p show-mappings (9b2e489)

Bug Fixes

studio: sort nodes in language echart (11b92f2)
accept single or multiple mappings in config.yaml (0c4961f)
always declare your file encoding, or Windows barfs (67de22f)
always declare your file encoding, or Windows barfs (56e327b)
avoid failure on corrupted pickles (6f30f0e)
catch same input and output g2p mapping bug (0a9d141)
correct fin diphthong mappings slightly (b8ae37d)
doh! always run the test suites before pushing your changes... (6af7d5e)
edit fin-ipa to eng-ipa mappings to fix some vowels (19a9189)
ensure python g2p/mappings/langs/init.py can always run (ea45afa)
find language_name robustly (4e0ccb9)
g2p show-mappings -v show in, out, rest, in that order (da06abd)
generated mappings should prevent feedback and apply-longest-first (d10944a)
grammar (e14cbcd)
lock click==8.0.4 since we support Python 3.6 (f941c64)
make maketokenizer disambiguate inlang and tok_path (9e7f986)
make Mohawk tokenizer recognize colon as a letter (767fed4)
make Mohawk tokenizer recognize colon as a letter (a32a3a7)
make und work in g2p studio (0dd3e25), closes #165
mic "o" didn't get mapped to proper eng-arpabet (1373b41)
name g2p in package.json, not readalongs (b18eea8)
recreate langs.pkl to allow merging (387c6fc)
remove stray BOMs everywhere (3c0f13b)
supported renamed dolgo/dogol distance in panphon (0c73399)
indices: handle orphan characters with heuristic of attaching to index of previous character if it exists, otherwise attaching to the index of the following character, if no characters exist before or after, then none type is returned. Fixes #172 (b1ca2cb)
moe: add self-mappings for k, m, n, p, s, t (a82d098)
regenerate mappings and configs (0d71872)
remove UTF-8 BOM and CRLF (will fix code separately) (5806ab9)
rules with alternations should tokenize correctly (9fd6407)
show default directory in help (679ffe1)
tell user to rerun g2p update (they can) (eeac7dd)
tliequiv and tceequiv had BOMs, update to remove them (5b5d0b5)
use an automatically generated mapping for moe-ipa -> eng-ipa (1bc9b05)
ci: trailing space for json (a58c205)
git: make fixes to ejective mappings (27003e2)
indices: fix numerous errors within the indices functionality (7a6c631)
mappings: fix normalization issues in win and eng mappings (fd2bb0f)
moe: remove two more duplicate rules (bd36902)
studio: updated reverse initialization and rule ordering values (2c9c61f)
win: use the \u02D0, not :; use prevent-feeding (80f80da), closes #100
update mappings (1cba84f)
use longest mapping for fin (12dc3da)
warn of missing language_name before caching (3eb0ed7)
write compact json rules with in+out first, then rest (e9cb0e6)

Performance Improvements

dockerfile: bump the OS to bullseye and optimize the build (20f1926)
test: speed up test_studio by minizing keyup events (4dad4d5)

Reverts

revert accidental removal of moe generated mapping (e11db2f)

Build Systems

Dockerfile should update pip before using it (3ca1ea2)
move flake8 config to setup.cfg (b637ac8)

Continuous Integration

add CI job to run tests on Windows (f4e5dc3)
adjust coverage configuration for CI and manual testing (907e4f1)
before a release, do full os/pyversion matrix testing (e01319f)
publish with build instead of calling setup.py directly (a1a024f)
quote python versions so 3.10 works (528ea8c)
run full matrix CI only on PRs to release (888404c)
run the CI tests on pull requests too (bb88b9b)
skip CI tests for commits with #no-ci in the message (5393045)
tests workflow should update pip (49abd45)
test-studio: adjust to test-studio CI workflow (bb03712)

Tests

tokenizer: more unit testing for refactored tok (119d2b7)
fix typo testing createmultimapping (3d7d23d)
show-mappings: unit testing for show-mappings (36f4996)
add eng-ipa and eng-arpabet tests (e5ed8b8)
add fin tests (6c608af)
allow testing update on input/output directories (eacd5de)
exercise win ejectives being removed from eng (6a69f60)
fix fin test (3639f69)
hopefully fully test everything (699ba10)
proper e2e tests for studio (5692a93)
silence expected warnings by asserting them (5fd2be8)
update fin tests (9ac849f)
studio: make the test_studio.py test suite more robust (303b794), closes #195

Documentation

add citation (7416775)
add citation to table of contents (1713e0c)
add CITATION.cff file (a9abc07)
add minimal coding docs to init.py (8593aa2)
add plain text citation and update name (bf5889c)
add See also section in README.md TOC (517b190)
fix help text for g2p update -o (c23fcf3)
let run_studio.py log a bit on screen where it's launching (2e38153)
update docstring (2ab8313)
update to stable version and link to blog (6478537)

Styles

add pre-commit hooks for black, etc and commitlint (6297bee)
blackify all Python files (9c26df4)
blackify and isort a few more files (cd31657)
blackify g2p/mappings/tokenizer.py, and globals should be lowercase, not uppercase (94561df)
blackify testcreatemapping.py (460deb0)
bump dev dependencies (93ea50f)
convert moetoipa.json to our new, compact format (9545832)
isort all Python files (1fa3d9d)
rewrite undtoipa.json in new compact format (95e25af)
show-mappings: black my work, and an extra test (4738921)
remove husky commitlint (replace by python gitlint) (e40a06f)
use gitlint to enforce conventional commits (29c4b05)
use gitlint to validate commit messages (904fb4c)

Code Refactoring

tokenizer: get_tokenizer still works, with deprecated warning (d6aa3ab)
tokenizer: refactor to same import strategy as make_g2p (24557b7)
apply a lot of flake8 recommendations (1bf8ae0)
black and isort and mypy, oh my (ac4ce57)
clean up imports (df56d8f)
clean up normalize with indices code (9a5bc3c)
factor out loading public test data (6cc278e)
further readability refactorizations (2b7d777)
general refactoring (bea2c09)
make test_fallback unit test easier to read (940af15)
spruce up distance metric code (876021e)
ci: move tests out of suite because they require 3.8 (cff8d1e)
show-mappings: -v as synonym for --verbose (f18220f)
show-mappings: remove old mapping viz generator (57b5aa1)
show-mappings: suggested edits (95653ed)
move cache_langs to utils (1aef962)
replace function attributes by module globals (e5199f0)
use else block (85fad34)

- Python
Published by github-actions[bot] over 3 years ago

g2p - Release v0.5.20220318

0.5.20220318 (2022-03-18)

Features

new g2p generate-mapping --from --to mode - WIP (3c9f3a9)
gen-map: implement and test gen-map with multiple target mappings (67dac09)

Bug Fixes

api: add index and debugger flags to documentation, add localhost server option and fix tests (fcc5225)
remove unused import (f573038)
docs: fixed typo in swagger spec (c6d8ea7)
test: fix coverage drop (bae0380)
move temporary test output to tmpdir for gen-map (ce95d48)
gen-map: allow --from and --to to alternatively be comma separated (24b686d)
gen-map: fix obsolete semicolon reference in error message (2d9facf)
gen-map: new generated mappings default to NFC (af8ca55)
gen-map: several improvements polishing the from/to mode (c0eb5f0)

Documentation

gen-map: better usage docs for --from/--to mode (746fce0)

Styles

apply some pylint recommended changes (7bd0934)
configured isort and mypy like in ReadAlongs/Studio (a346f05)
rewrite all generated JSON mapping in human-readable format (d7401f4)

Code Refactoring

output mappings in a more compact JSON format (204d8c5)

Tests

gen-map: improve unit testing coverage (5522a41)
gen-map: unit testing for new --from/--to gen-map (b25b006)
scan: make sure g2p scan works with NFC and NFD input (de2c09e)

- Python
Published by github-actions[bot] almost 4 years ago

g2p - Release v0.5.20211217

0.5.20211217 (2021-12-17)

Bug Fixes

deployment: move back to eventlet but lock master branch commit for gunicorn until new release (223390a)
deployment: replace eventlet with gevent (7cd6bfa)
deps: use locked gunicorn commit with a syntax that enables caching (333f023), closes /github.com/benoitc/gunicorn/pull/2581#issuecomment-994198667
fra: use \b for end of word, so it works before punctuation (22c9fd9)
mappings: reverse length sort abbreviations to prevent substring errors. fixes #133 (c7538c6)
test: test suite corrected for exceptions.IncorrectFileType change (cc64588)
tokenizer;--config: fix case-insensitive tok bug; --config can now load single mapping (3d88a6d)
fix bug causing exception with empty rules (4d644a0)
transducer: hide dummy rules (ad32663)
transducer: include all rules to debugger (d516064)
friendlier error messages when mapping or abbrev files not found (6226d4f)

Tests

add case feeding test mapping with test case (993ede7)
unit testing for bug causing exception with empty rules (9f1eecb)

Continuous Integration

bump rtd to 3.8 (0ff7b4b)
change from travis to gh-actions (75f8c46)
show github workflow build status badge (5d0c97b)
stop pointing to file (3a2f1f7)
switch to codecov (2e00437)

Styles

blackify mappings/init.py testmappings.py and testtransducer.py (d6ae834)
blackify mappings/utils.py and tests/testzlocal_config.py (9b156b8)

Documentation

coverage: change to codecov (3f91397)
readme: add link to blog (823672d)

Code Refactoring

reqs: move requirements to folder (5a606e1)
make_g2p should raise more meaningful exceptions on caller errors (adcf91f)

- Python
Published by github-actions[bot] about 4 years ago

g2p - Release v0.5.20211029

0.5.20211029 (2021-10-29)

Features

g2p generate-mapping --merge option (fixes #61) (55c1f08)
make the CLI command "g2p generate-mapping" more flexible (issue #61) (9ecec71)
config: add external config file option for cli (e4bccdc)

Bug Fixes

update travis CI URL (16275ef)
doctor: oops, that last PR broken "g2p doctor" with no argument (7483d48)
fra: fix fra->ipa to map all French characters (1483dea)
test: test local config last, since it has side effects (86d8220)
use is_ipa everywhere to detect IPA mappings, not .endswith("-ipa") (04d2b4b)
studio: fixed config table creation (33a8a58)

Performance Improvements

defer some expensive imports and initializations (2948537)
optimize Dockerfile to better use the Docker cache (9638e11)
optimize g2p generate-mapping by caching reused values, 8x faster (0e9ee3b)

Tests

fra: add some NFC test cases, to my existing NFD ones (7ac73c5)
fra: test the NFD cases to eng-ipa and eng-arpabet (e3902d3)
scan: re-enable testscanfra() now that fra is fixed (37fe1e1)

Continuous Integration

detect when g2p update is needed (d466c49)
detect when g2p update is needed, take 2 (3ba6127)

Code Refactoring

createipamapping.py - stay optimized, but easier to read (1135908)
delete unused p2p/init.py file/module (0d141b0)

Documentation

update README.md for new generate-mapping option (#61) (b52bc0c)

Styles

apply a few pylint recommendataions (b3489f2)
isort, black and pylint the files in the previous commit (ce0a4b1)

- Python
Published by github-actions[bot] over 4 years ago

g2p - Release v0.5.20210825

0.5.20210825 (2021-08-25)

Features

und -> und-ascii mapping calls text_unidecode.unidecode() (25cdf06)

Bug Fixes

deps: make g2p compatible with Flask 2.0.1 (1f8a9b2), closes #111
ikt: syllabic ᕼ is sometimes used instead of ASCII H (ee5d0a4)
moh: add plain h rule (1ee20d5)
tli: remove obsolete -norm- infix in tli-ipa to eng-ipa mapping name (d2fb7f3)
unit tests must use windows compat file joining (746f98a)
windows compat required declaring utf8 when opening files (e883938)

Performance Improvements

test: disable slow and ineffective testipaknownsegsall() (b708bbe)
test: remove slow ineffective test from testdoctor in testcli.py (ac38dba)

Styles

isort run.py to ease finding which tests are missing (22f0c4e)
remove superfluous whitespace at line ends (13da44e)

Documentation

better warning messages when g2p conversion check fails (2d1b89c)

Code Refactoring

clean up the UnidecodeMapping code (44a878f)
replace unidecode (GPL only) by text_unidecode (Artistic license) (288d79e)

Continuous Integration

add a check to make sure we don't introduce GPL dependencies (ea3f53d)
GPL test was not working quite right (716e540)

Tests

adjust unit testing for changed haa mapping structure (e11fa57)
migrated expensive doctor tests to testdoctorexpensive.py (f1aa1ab)
und: better coverage in und unit testing (6960f3f)

- Python
Published by github-actions[bot] over 4 years ago

g2p - Release v0.5.20210519

0.5.20210519 (2021-05-19)

Bug Fixes

moh: fixed mohawk equivalencies for low tones missing length marker (60f7432)

- Python
Published by github-actions[bot] almost 5 years ago

g2p - Release v0.5.20210514

0.5.20210514 (2021-05-14)

Features

augment panphon prepro with voiceless and tone markers (adce152)
better messages in is_panphon (16f3607)
check() implemented for tokenizing transducer (822e198)
ispanphon and isarpabet util methods (f0ca3f3)
is_panphon to apply our panphon preprocessor first (20e2aee)
is_panphon() to issue more warnings to help the user fix things (ddc6ed2)
temporarily make is_panphon() display how Panphon parses words that are not IPA (b53ee46)
WIP g2p convert --check option (e51468f)
check: displaywarnings arg to transducer.check() and ispanphon() (0bf00c5)
moh: added context sensitive phonological rules to moh (5c9dd5e)
moh: added context sensitive phonological rules to moh (b1e37b1)
moh: added new festival compliant mapping to moh (0f72c61)
moh: added new festival compliant mapping to moh (4200ad5)
moh: added reversible mappings for mohawk (778256d)
panphon_preprocessor: filter out primary stress mark, \u02c8 (ˈ) (ac73954)
panphon_preprocessor: strip all tone accents and bars for going to eng-ipa (34a89f8)

Bug Fixes

correct git merge error (073fdae)
correct git merge error and rerun g2p update (90a2c36)
g2p convert --tok still outputs two spaces after arpabet words (91f0def)
package-lock.json should not be committed (903a419)
crl+crj: support accented vowels as equivalent to double vowels in crl and crj (2769668)
fra: tidy up generated fra-ipa -> eng-ipa (e1de937)
haa: panphon is picky, use \u0261, not g (f407ae9)
haa: Panphon is picky, voiceless marker should go below the letter (6c69784)
haa: use \u02BC for ejective, not ', to please panphon (2557c76)
is_panphon: suppress spurious warnings about non-ipa characters (ce782c8)
lml: several corrections to lml (6203194)
moh: fixed low tone equivalencies and reordered ipa rules for reversibility (70e05f9)
moh: fixed moh specific test in transducer unittest (acef437)
moh: update mohawk mappings (8bca18f)
oji: use \u02D0 for vowel length, not ascii : (7c2c9d4)
reverse: change reverse feature to disregard context (ecfc30f)
studio: fix eventlet version (0c91bad)
tce: use \u02BC for ejective, not ', to please panphon (df00e19)
tli: use \u02BC for ejective, not ', to please panphon (2b6d148)
ttm: use \u02BC for ejective, not ', to please panphon (7d2264d)
remove g->\u0261 from panphon preprocessor; instead, we issue a warning about it (f5fe151)
transducer check should return True for cases with no known checks (4741f94)

Performance Improvements

more singleton speed testing work (153f04c)
optimize loading panphon distance with a singleton (4285759)
script to measure speed of different panphon.distance.Distance() init solutions (f4ace7e)
settle for the fastest singleton option (d97f790)
make_g2p: cache transducers since we make them over and over again (507c6f7)

Reverts

Revert "fix(haa): Panphon is picky, voiceless marker should go below the letter" (3673b04)

Styles

rename check test suite more suitably (3febc02)

Code Refactoring

even simpler, and slightly faster, singleton implementation (98ffee8)
make some parallel code structures more explicit (bc679a9)
simplify logic (9e49033)
simply logic (30a70c7)
Use simpler Singleton pattern implementation (2b66da9)
check, tokenizer: use is_ipa() instead of endswith("-ipa") (0bf5837), closes #102
moh: replace H and L with 1 and 2for festival format, change stress to high tone (7ce445e)

Tests

--check option with tokenizing transducer (4a0a20d)
add check ipa arpabet do dev test suite (fb1aed7)
add reverse tests for equiv (3dc0369)
adjust for arpabet producing trailing space (3122104)
default to source=g2p in tests/.coveragerc (7290c03)
default to source=g2p in tests/.coveragerc (1321e42)
more test cases for check ipa and arpabet (a433529)
public/data/ikt.psv (forgot to include in previous commit) (ec0fd04)
run check on all test data in public/data via test_cli.py (771634b)
tau cases going to eng-ipa and eng-arpabet (0b877ef)
unit test case for three hop tokenizer (a9eb917)
check: add test cases for display_warnings (b25af73)
srs: more srs test cases including arpabet checks (e97e654)

Documentation

Add link to PyPI releases to README.md (ff8b5fe)
link to github contributors page (f3538c3)
write up the g2p workflow in more details (f4056eb)

Continuous Integration

add automatic changelogs and change trigger (fe6be6b)
tag the repo last, so other steps still happens if it fails (72db514)
tag the repo with release version (8d11996)
temporarily fix Flask to 1.0.2 until flask-restful is compatible with 2.0.0 (62d4657)
temporarily fix Flask to 1.0.2 until flask-restful is compatible with 2.0.0 (bdb7d14)
testing tagging the repo on release (f96cb13)
update pythonpublish.yml to do only release br (ee6976c)

- Python
Published by github-actions[bot] almost 5 years ago

Recent Releases of g2p

g2p - v2.2.2

:bug: Bug Fixes

:whitecheckmark: Tests

g2p - v2.2.1

:sparkles: New Features

:bug: Bug Fixes

:recycle: Refactors

:whitecheckmark: Tests

:wrench: Chores

g2p - v2.2.0

:sparkles: New Features

:bug: Bug Fixes

:zap: Performance Improvements

:recycle: Refactors

:whitecheckmark: Tests

:wrench: Chores

g2p - v2.1.1

:sparkles: New Features

:bug: Bug Fixes

:zap: Performance Improvements

:recycle: Refactors

:whitecheckmark: Tests

:wrench: Chores

g2p - v2.1.0

:boom: BREAKING CHANGES

:sparkles: Major New Features

:sparkles: New Features

:bug: Bug Fixes

:zap: Performance Improvements

:recycle: Refactors

:whitecheckmark: Tests

:wrench: Chores

g2p - v2.0.0

:boom: BREAKING CHANGES

:sparkles: New Features

:bug: Bug Fixes

:zap: Performance Improvements

:recycle: Refactors

:whitecheckmark: Tests

g2p - Release v1.1.20230822

1.1.20230822 (2023-08-22)

Features

Bug Fixes

Performance Improvements

Tests

Build Systems

Continuous Integration

Documentation

Code Refactoring

g2p - Release v1.1.20230511

1.1.20230511 (2023-05-11)

⚠ BREAKING CHANGES

Features

Bug Fixes

Documentation

Continuous Integration

Tests

Code Refactoring

g2p - Release v1.0.20230417

1.0.20230417 (2023-04-17)

Bug Fixes

Code Refactoring

Continuous Integration

g2p - Release v1.0.20230412

1.0.20230412 (2023-04-12)

⚠ BREAKING CHANGES

Features

Bug Fixes

Documentation

Tests

Code Refactoring

Styles

Continuous Integration

g2p - Release v1.0.20230228

1.0.20230228 (2023-02-28)

Bug fixes

g2p - Release v1.0.20230224

1.0.20230224 (2023-02-24)

Features