Question on GEMV_AWQ use in AdaLN-Zero's linear (FluxSingleTransformer)

[kyohmin]

Hello,
While reviewing the code, I noticed that AdaLN-Zero's linear in FluxSingleTransformer uses GEMV_AWQ. Could you share the rationale for choosing this backend? Also, in:

GEMV_AWQ::GEMV_AWQ(int in_features, int out_features, bool use_bias, Tensor::ScalarType dtype, Device device)
    : in_features(in_features), out_features(out_features), group_size(64), lora_rank(0), lora_scale(1.0f),
      device(device) {
    this->qweight = Tensor::allocate({out_features / 4, ceilDiv(in_features, 8) * 4}, Tensor::INT32, device);
    this->wscales = Tensor::allocate({ceilDiv(in_features, group_size), out_features}, dtype, device);
    this->wzeros  = Tensor::allocate({ceilDiv(in_features, group_size), out_features}, dtype, device);
    this->bias    = use_bias ? Tensor::allocate({out_features}, dtype, device) : Tensor{};

    // !!! lora layout is different from w4a4 !!!
    this->lora_down = Tensor::allocate({lora_rank, in_features}, dtype, device, true);
    this->lora_up   = Tensor::allocate({out_features, lora_rank}, dtype, device, true);

    registerParams(qweight, "qweight", ParamFlags::LazyLoad)(wscales, "wscales")(wzeros, "wzeros")(bias, "bias")(
        lora_down, "lora_down", ParamFlags::Optional)(lora_up, "lora_up", ParamFlags::Optional);
}

what is the high-level purpose of / 4, * 4, and ceilDiv(..., 8)?

Thanks in advance,