[Feature Request] Select quantization via modifier arguments

[kylesayrs]

Background

Observers are a common abstraction used in quantization lifecycles used to collect information about observed weights/activations. Different observers can have different tradeoffs, for example, MSE observer often leads to better accuracy recovery, but takes significantly longer to run.

Purpose

• Right now, it is very difficult to select which observer to use for a given oneshot application. The user is forced to create a custom quantization config, and specify observers at the scheme level.

Example of old workflow

recipe = [
    QuantizationModifier(
        config_groups={
            "group_0": {
                "targets": ["Linear"],
                "weights": {
                    "num_bits": 4,
                    "type": "float",
                    "strategy": "tensor_group",
                    "group_size": 16,
                    "symmetric": True,
                    "observer": "mse",
                },
                "input_activations": {
                    "num_bits": 4,
                    "type": "float",
                    "strategy": "tensor_group",
                    "group_size": 16,
                    "symmetric": True,
                    "dynamic": "local",
                    "observer": "mse",
                },
            }
        },
    )
]

• It would be much nicer if a user could specify a recipe like this:

recipe = [
    QuantizationModifier(scheme="NVFP4", weight_observer="MSE", input_observer="MSE")
]

Or like this

recipe = [
    QuantizationModifier(scheme="NVFP4", observer={"weights": "MSE", "input": "MSE"})
]

Requested Changes

• Add an "observer" or "weight_observer/input_observer" argument to QuantizationMixin, allowing users to specify observers easily from the recipe
• Please include tests to verify your changes

[hshen14]

Nice proposal, @kylesayrs ! The new API will simplify the code and help improve the UX of quantization for the most cases! Do we still need to provide the comprehensive configs for advanced users?

[kylesayrs]

Hi @hshen14! Yes, the idea here is that QuantizationModifier (or any modifiers which inherit from QuantizationMixin) will provide convenience arguments such as scheme="W4A16", ignore=[...], observers=..., but will maintain the config_groups argument for more fine-grained and advanced control of the quantization config.