The downside is... that it's Chinese? BYD products are way more proven than any EV technology America has ever touched. I think you're just racist.
v_pp
joined 4 years ago
Not really, you still need to have a lot of tooling for compilers, hardware, etc. But it is an area where China has an opportunity to both eliminate dependence on proprietary foreign technology, and develop a technology that nobody else has a real edge in
AI/ML research has long been notorious for choosing bullshit benchmarks that make your approach look good, and then nobody ever uses it because it's not actually that good in practice.
It's totally possible that there will be legitimate NLP use-cases where this approach makes sense, but that is almost entirely separate from the current LLM craze. Also, transformer-based LLMs pretty much entirely supplanted recurrent networks as early as like 2018 in basically every NLP task. So even if the semiconductor industry massively reoriented to producing chips that support "MatMul-free" models like this one to even get an energy reduction, that would still mean that the model outputs would be even more garbage than they already are.
I'm highly skeptical of this at first glance. Replacing self-attention with gated recurrent units seems like a decisive step back in natural language processing capabilities. The advancement that gave rise to LLMs in the first place was when people realized that building networks out of a bunch of self-attention blocks instead of recurrent units like GRU or LSTM was extremely effective.
In short, they are proposing an older type of model which are generally outclassed by attention-based transformers that power all the LLMs we see today. I doubt it will be able to achieve nearly as good results as existing LLMs. I foresee this type of research being used to silence criticisms of the ungodly amounts of energy used by LLMs to say "See, people are working on making them way more efficient! Any day now..." Meanwhile they will never come to fruition.
This is some absolutely depraved shit. You're sitting here justifying levels of death and destruction and human misery that are beyond your comprehension just because of some made up conspiracy theories about Russia "meddling" with other countries. In what possible universe does that make you anything other than pure fucking evil?
so, what, you just make shit up then deflect when someone points that out?
So what I'm hearing is that the Chinese are better at calculus than the Americans, sounds about right.
Although the article doesn't get into any detail, it sounds like what they're doing is applying similar optimization techniques to those that have enabled the past 15 years of AI/ML development in order to optimize their aircraft design. It's not surprising that China was able to do that; there's been an enormous amount of work into developing math, software, and hardware for that stuff. But what's more surprising is that the US hasn't implemented it already. I guess the X-47B program was halted right as the ML/deep learning explosion started so these teqniques weren't really widespread yet. But I find it hard to believe that the US hasn't done it already.