They were able to efficiently encode visual information to be used by further networks. In this case the further network was a language model trained on an OCR task.

The news is the technique, the OCR software is a demonstration of the technique. Encoding visual information efficiently is also key for robotics which use trained networks in their feedback control loops. Being able to process 10 times as much visual data with the same hardware is a very significant increase in capability.

[–] Moidialectica@hexbear.net 2 points 2 weeks ago

it doesn't actually process text, which is why it's more efficient, it can essentially take in ten times the text through images without suffering the penalties associated by having that many tokens