Technology

It's not as accurate as you'd like it to be. Some issues are:

• It's quite lossy.
• It'll do better on images containing common objects vs rare or even novel objects.
• You won't know how much the result deviates from the original if all you're given is the prompt/conditioning vector and what model to use it on.
• You cannot easily "compress" new images, instead you would have to either finetune the model (at which point you'd also mess with everyone else's decompression) or do an adversarial attack onto the model with another model to find the prompt/conditioning vector most likely to create something as close as possible to the original image you have.
• It's rather slow.

Also it's not all that novel. People have been doing this with (variational) autoencoders (another class of generative model). This also doesn't have the flaw that you have no easy way to compress new images since an autoencoder is a trained encoder/decoder pair. It's also quite a bit faster than diffusion models when it comes to decoding, but often with a greater decrease in quality.

Most widespread diffusion models even use an autoencoder adjacent architecture to "compress" the input. The actual diffusion model then works in that "compressed data space" called latent space. The generated images are then decompressed before shown to users. Last time I checked, iirc, that compression rate was at around 1/4 to 1/8, but it's been a while, so don't quote me on this number.

edit: fixed some ambiguous wordings.

You can hardly consider it compression when you need a compute expensive model with hundreds of gigabytes (if not bigger) to accurately rehydrate it

If you ignore the fact that the generated images are not accurate, maybe.

They are very similar so they are infringing but nobody would use this method for compression over an image codec

I was thinking about this back when they first started talking about news articles coming back word for word.

There's no way for us to tell how much of the original data even in a lossy fashion can be directly recovered. If this was as common as these articles would leave you to believe you just be able to pull anything you wanted out on demand.

But here we have every news agency vying to make headlines about copyright infringement and we're seeing an article here and there with a close or relatively close result

There are millions and millions of people using this technology and most of us aren't running across blatant full screen reproductions of stuff.

You can tell from some of the artifacts that they've trained from some watermark images because the watermarks kind of show up but for the most part you wouldn't know who made the watermarking if all the watermarking companies didn't use rather unique patterns.

The image that we're seeing on this news site of the joker is quite exceptional, even from a lossy standpoint, but honestly it's just feeding the confirmation bias.

Holy shit I didn't even think about that.

Essentially the model is compressing the image into a prompt.

Instead of the bitmap being 8MB being condensed down into whatever the jpeg equivalent is, it's still more than a text file with that exact prompt that gave.

Optimal tip-to-tip efficiency has been achieved.

Chat GPT it's over 500 gigs of training data plus over 300 gigs of RAM, and Sam Altman has been quite adamant about how another order of magnitude worth of storage capacity is needed in order to advance the tech.

I'm not convinced that these are compressed much at all. I would bet this image in its entirety is actually stored in there someplace albeit in an exploded format.

Compression is actually a mathematical field that's fairly well explored, and this isn't compression. There are theoretical limits on how much you can compress data, so the data is always somewhere, either in the dictionary or the input. Trained models like these are gigantic, so even if it was perfect recall the ratio still wouldn't be good. Lossy "compression" is another issue entirely, more of an engineering problem of determining how much data you can throw out while making acceptable compromises.

Results vary wildly. Some images are near pixel perfect. Others, it clearly knows what image it is intended to be replicating. Like it gets all the conceptual pieces in the right places but fails to render an exact copy.

Not a very good compression ratio if the image you get back isn't the one you wanted, but merely an image that is conceptually similar.

I mean, only if you have the entire model downloaded and your computer does a ton of work to figure it out. And then if any new images are created the model will have to be retrained. Maybe if there were a bunch of presets of colors to choose from that everyone had downloaded and then you only send data describing changes to the image

I made a novel type of language model, and from my calculations after about 30gb it would cross over an event horizon of compression, where it would hold infinitely more pieces of text without getting bigger. With lower vocabulary it would do this at a lower size. For images it's still pretty lossy but it's pretty cool. Honestly I can't mental image much better without drawing it out.