Skip to content

Commit b59c604

Browse files
committed
Add diverse text type benchmark with tokenization quality metrics
1 parent 6e36c37 commit b59c604

File tree

6 files changed

+1366
-2
lines changed

6 files changed

+1366
-2
lines changed

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,3 +164,7 @@ cache/
164164
*.mp3
165165
*.wav
166166
*.ogg
167+
168+
# Benchmark results and local environment
169+
langextract_env/
170+
benchmarks/benchmark_results

0 commit comments

Comments
 (0)