The top 3 repos sorted by stars on Huawei's github page are all for quantization. I am not an LLM scientist so I can't tell what the difference is between them, or why they need three different pipelines for model quantization.
SINQ was created last year. It was for weights only and it was calibration-free quantization. Then KVarN is for KV-cache quantization to get longer context. MACaQ is multimodal, activation and weights calibrated quantization. 3 different projects
Why does some HN users seem to prefer GitHub page whenever available at expense of official site? I mean, Huawei is still sanctioned, anything they host third party outside China can disappear at any hour.
I prefer GitHub, because I can see the code and immediately know where to click. Product pages are always full of various marketing shots, videos, etc that just takes more time to parse through.
This is pretty much the reason that if I read about a new product, I generally just go to the Wikipedia page. I just want the no-bullshit version, no "Say hello to blah" video or marketing word salad.
29 comments:
People are reproducing it and creating GGUFs https://www.reddit.com/r/LocalLLaMA/s/36atJ2nNjh
The results shown there are very good, much better than for the previously widely used quantizers.
Nice for Huawei to share it.
The top 3 repos sorted by stars on Huawei's github page are all for quantization. I am not an LLM scientist so I can't tell what the difference is between them, or why they need three different pipelines for model quantization.
SINQ was created last year. It was for weights only and it was calibration-free quantization. Then KVarN is for KV-cache quantization to get longer context. MACaQ is multimodal, activation and weights calibrated quantization. 3 different projects
https://huawei-csl.com/blog/macaq/
Why does some HN users seem to prefer GitHub page whenever available at expense of official site? I mean, Huawei is still sanctioned, anything they host third party outside China can disappear at any hour.
I prefer GitHub, because I can see the code and immediately know where to click. Product pages are always full of various marketing shots, videos, etc that just takes more time to parse through.
This is pretty much the reason that if I read about a new product, I generally just go to the Wikipedia page. I just want the no-bullshit version, no "Say hello to blah" video or marketing word salad.
In this particular case I think you'd be pleased with it https://huawei-csl.com/blog/macaq/
It is from their lab in Switzerland, after all.
Looks like many of their repos there have been around for a while. Also not sure why we would sanction free stuff from them
Well I'm not a lawyer but are you allowed to provide material support to sanctioned entities?
It’s a commerce department sanction not a treasury department sanction (which would be far more serious).
Did they release any model? Seems good for Strata
Qwen 3.8 27B has a macaq quant available
What a project name.
Probably intended to reference macaque monkeys.
I forgot my French, but isn't it incredibly rude?
Why?
I think the immature commenters are referring to the pronunciation being similar to "my cock"
Aaa, thanks I didn't think of that.
Congratulations on not being a total degenerate.
It's also a racial slur in certain languages.
It's also "Roach" in Hebrew (מקק)
What are those?
No, no, not that.
I'd assume the French know something about naming projects in the vein of Coq.
xD
First thing my mind went to.
"Huawei releases MACaQ"
"Just like Jensen released his..."
I do not think that macaques would mind.
Just enough plausible deniability :P