Use Google's Skia library over sdl-gpu for vector graphics rendering

[colbyn]

TL;DR: I propose that the UIKit-cross-platform project adopt Google's Skia library for its graphics rendering needs instead of using SDL. Skia's support for modern APIs like Vulkan, alongside its proven performance in UI-centric environments (like Jetpack Compose), makes it the most future-proof and strategic choice for this project.

Key Reasons to Use Skia Over SDL-GPU:

1. OpenGL ES is Becoming Obsolete on Android:
   OpenGL ES, which SDL_gpu primarily relies on, is gradually being phased out on Android in favor of Vulkan. Google's focus is now on Vulkan as the preferred graphics API due to its superior performance, reduced overhead, and better support for multi-threading. Skia's native support for Vulkan allows UIKit-cross-platform to leverage modern graphics capabilities on Android without being tied to outdated technologies, ensuring long-term compatibility and performance.

2. Proven Industry Standard for UI Rendering (Used by Jetpack Compose):
   Skia is already the graphics engine behind Jetpack Compose, Android's modern UI toolkit. This shows its effectiveness and reliability in rendering high-performance, smooth UIs at scale. By using the same technology as Jetpack Compose, UIKit-cross-platform can achieve a similar level of graphical fidelity and efficiency, providing a seamless user experience on Android.

3. High-Level Abstraction and Rich Feature Set:
   Skia's higher-level API simplifies UI development by offering built-in support for crucial features like sub-pixel anti-aliasing, vector graphics, hardware-accelerated compositing, and image filtering. This approach reduces development complexity compared to SDL's OpenGL-based rendering, which requires more low-level code to achieve the same results. With Skia, the focus can remain on building high-quality UI components rather than managing graphics pipelines.

4. Cross-Platform Flexibility and Modern API Support:
   Skia supports multiple backends, including Vulkan, Metal, OpenGL, and Direct3D, making it highly versatile across different platforms. As graphics technology continues to evolve, this flexibility ensures that UIKit-cross-platform can adapt easily to new trends without requiring major architectural changes. In contrast, SDL_gpu's focus on OpenGL limits its ability to integrate seamlessly with these newer APIs, making it less future-proof.

5. Performance Optimization Tailored for UI Workloads:
   Skia is specifically optimized for rendering interactive user interfaces and handling complex text and graphics operations. Its pipeline is designed to make the most of hardware acceleration, which is essential for smooth animations and responsive touch interactions. While SDL_gpu can be efficient for general-purpose 2D graphics, achieving the same level of optimization for UI-heavy scenarios would require considerable custom development.

6. Wide Adoption and Active Development Community:
   Skia is backed by Google and used in major products like Chrome, Android, and Flutter, ensuring it is well-supported and continuously optimized for modern use cases. The library benefits from a large developer community and frequent updates, keeping it aligned with the latest advancements in graphics technology. This strong ecosystem support provides a level of reliability that SDL cannot match.

---

Personally:

First of all, I love this project! I was just looking into building my own UIKit-like cross platform framework and then I discovered this project, although I was a bit dismayed to see that it uses an obsolete technology for the vector graphics rendering side of things. I’ve used Skia before and it’s amazing! Very simple and easy to use, and it supports a multitude of platforms among other things (see above points).

More broadly I was thinking, with regard to UIKit's dependencies, the dependency tree is roughly:

UIKit
 ├─ TextKit
 │   ├─ CoreText
 │   └─ CoreGraphics
 ├─ CoreGraphics
 │   ├─ CoreFoundation
 │   ├─ QuartzCore
 │   ├─ ImageIO
 │   └─ IOSurface
 ├─ Foundation
 ├─ QuartzCore
 ├─ CoreImage
 ├─ CoreData
 ├─ AVFoundation
 ├─ CoreAnimation
 ├─ Metal
 ├─ SceneKit
 ├─ SpriteKit
 ├─ CoreMotion
 └─ CoreLocation
CoreFoundation

(Although I'm mostly concerned with CoreGraphics, CoreText, and TextKit.)

Organizationally, shouldn’t you start with the lower-level frameworks and then work your way up to UIKit?

This is the direction I was going down before I noticed this project. As an amateur compiler engineer, I’ve been working on a parser and AST for the Objective-C and Swift Interface Files located under /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk/System/Library/Frameworks (thought this would be a good place to start and see what I can learn).

My ultimate plan was to use what public info I have on my Mac to bootstrap the project. Then I’d try building specialized tools to autogenerate as much of the API implementation as possible using AI text generation models (although as someone who's done similar work before I'm concerned about the cost)... After I’ve gone as far as I could with autogenerated code, my plan was to then open up the work for contributions from the community.

If there’s any interest on your end, I’d be interested in collaborating with ya’ll on this!

[ephemer]

Hey there. Thanks, I appreciate your enthusiasm and the energy you put into this.

Skia is something we looked at when we first created this project, at the time it was a software renderer only and did not support GPU backends. We have it as a TODO to add it as a dep to implement CGContext and other low level drawing primitives, but we've never actually needed it, so it was a nice-to-have.

I would need to look into Skia's GPU support more deeply but, in general, exchanging the renderer is something we'd very much like to do. More broadly, SDL has been a helpful backbone to this project, but over time it has become more of a hinderance than a help. But our text rendering is also intertwined with SDL due to sdl-ttf. So text rendering and layouting is something else that would either need to be rewritten with another library, or it would need to be supported by Skia.

For many years the planned approach to the above issues was to use bgfx as a low-level HAL and write the few primitives we need manually. That would have a smaller impact on bundle size and compile times: we thought we'd need to run this library on Wasm, so binary size was a top concern. But we went for a different approach there in the end. That approach is still an interesting alternative IMO, depending on compile times, platform compatibility, and binary size of Skia.

Regarding the structure, what you've posted is correct of course. For our project though, we did not want to replicate the minutiae of the framework's implementation details unless we relied on those details to actually do meaningful work. To do a true "port" we'd need to, but for us it's an explicit non-goal. We just want to write UIKit code that works on both platforms. Ideally we'd extend that picture to include SwiftUI as well, but as a small company we've had other priorities.

If you're interested to integrate Skia, please feel free to make a POC PR, even with some APIs commented out to begin with. With a working basis, it would be easier for us to assess how much work it'd be to complete the work and could also support if it seems feasible.

[ephemer]

Even though we don't explicitly create modules for CoreText and CoreGraphics, the project does expose some of their key functionality already.

What we weren't interested in was trying to recreate verbose C APIs with hundreds of possible combinations of parameters and code paths- we use max 1-2 of the available options and tried to build abstractions that allowed us to do without them.