Skip to content

[NPUW] integration of LPT-fp8 passes for optimizing fp8 kv-cache behavior #32448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

[NPUW] integration of LPT-fp8 passes for optimizing fp8 kv-cache behavior #32448

wants to merge 5 commits into from
+223 −23

Conversation

@esmirno
Copy link
Contributor

@esmirno esmirno commented Oct 16, 2025 •
edited
Loading

Details:

  • Applied specific LPT passes to decompose FakeConvert layer, and leaving kv-cache in fp8 precision.
  • performance regressions will be addressed in upcoming work.

Tickets:

  • E-186663

@github-actions github-actions bot added category: NPU OpenVINO NPU plugin category: NPUW NPUW plugin labels Oct 16, 2025
@esmirno esmirno changed the title initial iteration of lpt8 integration of NPUW initial iteration of lpt8 integration into NPUW Oct 16, 2025
@dmatveev dmatveev added this to the 2026.0 milestone Oct 31, 2025
@esmirno esmirno marked this pull request as ready for review November 13, 2025 13:46
@esmirno esmirno requested review from a team as code owners November 13, 2025 13:46
esmirno
esmirno commented Nov 13, 2025
View reviewed changes
src/plugins/intel_npu/src/plugin/npuw/llm_compiled_model.cpp
if (fcTypesInput.empty() || !fcTypesRemained.empty()) {
LOG_WARN("FakeConvert layers not decomposed - leaving kv-cache in " << kv_kache_storage_type
<< " precision");
} else if (fcTypesInput.size() > 1) {
Copy link
Contributor Author

@esmirno esmirno Nov 13, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if LPT passes applied and different precision detected - consider change message + leaving kv-cache in " << kv_kache_storage_type

esmirno
esmirno commented Nov 13, 2025
View reviewed changes
Copy link
Contributor Author

@esmirno esmirno left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tbd

@esmirno esmirno changed the title initial iteration of lpt8 integration into NPUW [NPUW] integration of LPT-fp8 passes for optimizing fp8 kv-cache behavior Nov 13, 2025
@dmatveev
Copy link
Contributor

dmatveev commented Nov 18, 2025

@AsyaPronina please have a look

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

@AsyaPronina AsyaPronina

Labels

category: NPU OpenVINO NPU plugin category: NPUW NPUW plugin

Projects

None yet

Milestone

2026.0

Development

Successfully merging this pull request may close these issues.

3 participants

@esmirno @dmatveev @AsyaPronina