Skip to content

Commit bd1e3d2

Browse files
authored
Add diverse text type benchmark with tokenization quality metrics (#272)
1 parent 284dae9 commit bd1e3d2

File tree

5 files changed

+1409
-0
lines changed

5 files changed

+1409
-0
lines changed

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,3 +164,7 @@ cache/
164164
*.mp3
165165
*.wav
166166
*.ogg
167+
168+
# Benchmark results and local environment
169+
langextract_env/
170+
benchmarks/benchmark_results

0 commit comments

Comments
 (0)