Experimental support for setdesktopsize / extendeddesktopsize

[maxnet]

When enabled with the -setdesktopsize option this allows
clients like novnc and tigervnc to request a change of
framebuffer resolution to fit that of the client window.

On the server this is implemented by leaving the physical
display resolution as-is, but setting a new framebuffer
resolution and scaling that framebuffer to the physical
display using RANDR calls.
As a result the view on the physical display may look a bit
distorted due to the scaling, but looks perfect on the remote
client. (While that is normally the opposite case)

When last client disconnects original framebuffer resolution
is restorted.

Known limitations:

• Implementation currently limited to computers with a single
  screen attached.

• Only works on X servers/drivers with RANDR support

• Does not always work properly on headless sytems with no monitor attached. (Needs some investigation)

• Needs a recent libvncserver with the setdesktopsize hook support recently merged.

Closes #91

[bk138]

Thanks! Before any further review: please make sure the code builds with an older LibVNCServer version, too.

[maxnet]

Fair enough. Added some #if's

[andrewmcewen]

Is there any update on this PR?

[ioMatrix]

Having this feature work would be a major plus. Please let me know what kind of support you require to get this working.

[bk138]

It would be super nice if someone could tackle and sort out the remaining known limitations maxnet mentioned so we not merge in something half-baked :-)

[Neustradamus]

This @maxnet PR is linked to one ticket but not merged yet:

• ExtendedDesktopSize support? #91

[Micket]

Code looks very good in my opinion, and it all is working fine for me.

1. Xvfb+x11vnc resizing works fine.
2. Xorg+dummy driver is a dead end. Just ignord XRRSetScreenSize, and I suspect it's not properly populating crtc information, so using it XRRSetCrtcConfig might be one cause of crashes. Dummy driver isn't likely to ever see much love again, and there is no reason to use it; Xvfb works fine for softare rendering.
3. Xorg+nvidia. Resizing works fine. I'm doing this from a headless node, and I can't see any issues.

I vote for merging this patch as is. The limitations are not severe at all:

1. Single screen is the most useful case for a vnc session. I have in all my years never seen anyone use multi-screen VNC, and those who do would simply get no change in behavior. It's very hard to make sense of how multi-screen would even work here.
2. Not much of a limitation. Save to say the same applies to all who implement setdesktopsize.
3. I'm not sure what scenario it would fail on. It has been successful on the headless setups i've tested it on.
4. Long since merged.

I will be including this hook when I package x11vnc for EasyBuild, since I consider it a crucial feature, and this now allows me to run true Xorg remote desktops for HPC clusters with full hardware acceleration (GLX, EGL, Vulkan, all without involving VirtualGL). easybuilders/easybuild-easyconfigs#24662

[bk138]

@Micket are you maybe interested in #186?