Memory issues in IDocumentUrlService after upgrading to v17

[SelwynEdelbroek]

Which Umbraco version are you using?

17.0.0

Bug summary

We have upgraded a website from v8 to v17 for a client of ours. This website has a lot of nodes, containing many movies, games and series with ratings about if it is safe to watch or play for Dutch citizens. This dataset contains in and about 9 million urls:

Memory usage:

Before the website started with an average of 4 GB of memory on load a startup in v8. After upgrading to v16 or v17 we noticed that the IDocumentUrlService is caching all these urls segments.
While starting the website v16/v17 before fully booted the website uses up to 8GB of memory. When we uses a clean Umbraco v16 or v17 instance the memory usage is around 1 or 1.5gb with the IDocumentUrlService removed and referencing the database of the client. Since this upgrade we've noticed an big difference in memory load since this newer Umbraco version. This memory change is a problem for hosting services or platforms regarding out of memory issues when using a large website with several languages. We currently are experiencing slow Umbraco instances and or server due to the memory usage.

Specifics

This happens on startup of the v17 website inside the IDocumentUrlService, inside the init method the code starts fetching all DB entries and caches them inside a dictionary.

Steps to reproduce

We have an Umbraco v17 project that reproduces our scenario, this is included in the zip file. To check the Umbraco backoffice please check the appsettings inside the project.

Reproduce from scratch would be:

1. Create Umbraco instance with minimum of 10 to 20 languages
2. Add pages with inheritance and a node structure like blog posts

UmbracoMemoryUsage.zip

Expected result / actual result

The expected result: it would be nice that the in memory usage would not increase on startup.
The actual result: the Umbraco startup fetches all content of the referenced DB table and stores it in memory.

[github-actions]

Hi there @SelwynEdelbroek!

Firstly, a big thank you for raising this issue. Every piece of feedback we receive helps us to make Umbraco better.

We really appreciate your patience while we wait for our team to have a look at this but we wanted to let you know that we see this and share with you the plan for what comes next.

• We'll assess whether this issue relates to something that has already been fixed in a later version of the release that it has been raised for.
• If it's a bug, is it related to a release that we are actively supporting or is it related to a release that's in the end-of-life or security-only phase?
• We'll replicate the issue to ensure that the problem is as described.
• We'll decide whether the behavior is an issue or if the behavior is intended.

We wish we could work with everyone directly and assess your issue immediately but we're in the fortunate position of having lots of contributions to work with and only a few humans who are able to do it. We are making progress though and in the meantime, we will keep you in the loop and let you know when we have any questions.

Thanks, from your friendly Umbraco GitHub bot 🤖 🙂

[AndyButland]

Thanks for the report @SelwynEdelbroek. I've been looking this afternoon to see how we might optimise this in #21066

[SelwynEdelbroek]

@AndyButland thank you for looking into this so fast! If we can help by checking a RC version please let us know.