Hasn't this crossed over into being a philosophical question? You want to attest to reality or provenance. But then you start making lists of "acceptable" changes: file format; compression; color-correction; size; taking "medium" into account... It then slowly slides into "average human perceptibility"; "irrelevant details"; keeping the "spirit" of the image intact; judging the intention or motive of the user or of the viewer; aligning the image with "values"; appropriate/legal use... etc. Not sure what the end-game is?
We are trying to make images in the AI era be as reliable as they were pre-AI? But they were not reliable pre-AI, it was just more difficult to intentionally doctor them.
I feel like we were already in a weird gray area. Cellphone cameras use all kinds of programmatic tricks to make their output better than the exact information the lens captures. I think people just didn’t realize how much their phones were doing. Is that “fake”?
I think for photojournalism it should be original raw image, as captured in camera — zero edits for colour correction/cropping/filesize etc — platforms can link a digital sig from a postprocessed version but you keep the provenance proof tied to a published original.
I'm trying to think of recent real-world examples where the reality of the photo was of major importance. Photographs just don't seem to have the same significance, even as potential verification of their realness becomes more achievable.
Entering photos as evidence in a legal proceeding is currently bizarrely complex. You actually have to admit an expert witness to testify that the software used to process a photograph will not introduce artifacts. Even if you are simply zooming in or cropping an image.
There’s now even a wiki article on the usage of AI generated images in American politics, mostly dealing with the obvious slop that Trump has produced, rather than insidious edits of real images
> our tool can verify a large family of image transformations
Is this family large enough to transform real image A into arbitrary fake image B?
> ZK-JPEG can merge transparent or translucent layers into an image, useful for placing visual watermarks,
creating double exposures, or merging visual layers, potentially AI generated over portions of the image. In
our tool, the transparent layer is revealed, but anything beneath an opaque portion of the layer becomes
secret. Note that in the case of an AI generated layer, the layer itself is revealed, minimizing the dishonesty
in using generative AI (but not minimizing the dishonesty of stealing artwork to train the AI)
Seems like the answer to my question is "yes", so you have to carefully inspect the transformations to see if they look legit. (The parenthetical was a surprising inclusion in an academic context)
Presumably in a real application there would be a list of acceptable operations.
Like first you have to show you can do everything necessary in the system which is what this paper does. Step 2 is coming up with a policy of what restrictions to place on allowed transformations
If you're preserving the original, then you don't need the ZKP. Part of the appeal of this approach is that you could apply redactions or make the file smaller, without invalidating the signature.
17 comments:
Hasn't this crossed over into being a philosophical question? You want to attest to reality or provenance. But then you start making lists of "acceptable" changes: file format; compression; color-correction; size; taking "medium" into account... It then slowly slides into "average human perceptibility"; "irrelevant details"; keeping the "spirit" of the image intact; judging the intention or motive of the user or of the viewer; aligning the image with "values"; appropriate/legal use... etc. Not sure what the end-game is?
We are trying to make images in the AI era be as reliable as they were pre-AI? But they were not reliable pre-AI, it was just more difficult to intentionally doctor them.
I feel like we were already in a weird gray area. Cellphone cameras use all kinds of programmatic tricks to make their output better than the exact information the lens captures. I think people just didn’t realize how much their phones were doing. Is that “fake”?
I think for photojournalism it should be original raw image, as captured in camera — zero edits for colour correction/cropping/filesize etc — platforms can link a digital sig from a postprocessed version but you keep the provenance proof tied to a published original.
That's pretty much how the Apple one works. The raw sensor data is signed. But you have to process it because raw sensor data is unviewable.
Provide the raw original, and the exact processing steps used to get the presentable one.
no no no you need to buy a new phone!
/s
If this ever became widespread, people would game the "verified real" checkmark using the analog gap.
1. Print AI photo 2. Point fancy expensive camera at printed photo 3. Take a "real" photo
Then you'd see more sensors and even more expensive cameras. Then you'd see miniatured virtual-production volumes.
So, this is not a solution. It's just an arms race disguised as one.
Can you photograph a photo so it’s not really noticeable?
This would make deepfakes more expensive because you would have to tightly control the chroma and illuminance of the display.
I'm trying to think of recent real-world examples where the reality of the photo was of major importance. Photographs just don't seem to have the same significance, even as potential verification of their realness becomes more achievable.
Entering photos as evidence in a legal proceeding is currently bizarrely complex. You actually have to admit an expert witness to testify that the software used to process a photograph will not introduce artifacts. Even if you are simply zooming in or cropping an image.
An everyday scenario is Real estate photography and Facebook marketplace where people are having a field day AI faking photos.
Having a tick showing the photo is verified real would add a lot of trust to online platforms.
There’s been a few, but it’s becoming so common people have stopped noticing, but here’s a few recent ones:
https://www.bbc.co.uk/news/live/ce9yydgmzdvt?post=asset%3Aae...
https://www.france24.com/en/middle-east/20260417-press-assoc...
There’s now even a wiki article on the usage of AI generated images in American politics, mostly dealing with the obvious slop that Trump has produced, rather than insidious edits of real images
https://en.wikipedia.org/wiki/AI-generated_content_in_Americ...
Tie this into the Apple/Android/Sony/Leica signed photos, and you get provenance from capture to publish
Interesting work.
> our tool can verify a large family of image transformations
Is this family large enough to transform real image A into arbitrary fake image B?
> ZK-JPEG can merge transparent or translucent layers into an image, useful for placing visual watermarks, creating double exposures, or merging visual layers, potentially AI generated over portions of the image. In our tool, the transparent layer is revealed, but anything beneath an opaque portion of the layer becomes secret. Note that in the case of an AI generated layer, the layer itself is revealed, minimizing the dishonesty in using generative AI (but not minimizing the dishonesty of stealing artwork to train the AI)
Seems like the answer to my question is "yes", so you have to carefully inspect the transformations to see if they look legit. (The parenthetical was a surprising inclusion in an academic context)
Presumably in a real application there would be a list of acceptable operations.
Like first you have to show you can do everything necessary in the system which is what this paper does. Step 2 is coming up with a policy of what restrictions to place on allowed transformations
Viewers could provide a way to see the original before transforms, and perhaps apply each one at a time to see how the end result is arrived at.
If you're preserving the original, then you don't need the ZKP. Part of the appeal of this approach is that you could apply redactions or make the file smaller, without invalidating the signature.