Skip to content

[BUG]: Inaccurate duration metric from Ollama #4567

@jonathanortega2023

Description

@jonathanortega2023

How are you running AnythingLLM?

Docker (local)

What happened?

I'm setting up analytics in my fork and noticed tps/duration not lining up with expectations (especially when not streaming). In my tests with Phi4:14b streaming was about 60tps, but non-streaming was about 15tps. In either case, metrics are capturing duration and using that (with network latency + time to first token) to calculate outputTps, which is misleading.

Instead of using duration for the calculation, it should be done with eval_duration, as this is how output speed is generally calculated.

Metadata

Metadata

Assignees

No one assigned

    Labels

    possible bugBug was reported but is not confirmed or is unable to be replicated.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions