Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ATProto -> ActivityPub images are not transmitted. #1000

Closed
KDederichs opened this issue May 3, 2024 · 27 comments
Closed

ATProto -> ActivityPub images are not transmitted. #1000

KDederichs opened this issue May 3, 2024 · 27 comments
Labels

Comments

@KDederichs
Copy link

It seems that Bluesky posts that have images attached do not transmit them to Mastodon. All the image posts in my feed are just text :)

@snarfed
Copy link
Owner

snarfed commented May 3, 2024

Hmm, that's definitely worked before, and it worked in my tests. Here's an example: Bluesky post, bridged fediverse post with image is visible on @[email protected] 's timeline if you scroll down.

What's your Bluesky handle?

@KDederichs
Copy link
Author

Well I'm following that profile for example via Bridgy on Mastodon:
https://bsky.app/profile/shrimposaurus.bsky.social

The Posts are basically all with image but none of them show up (it's backfilling just the text)
Screenshot 2024-05-03 at 23 32 12

@snarfed
Copy link
Owner

snarfed commented May 4, 2024

Understood. Which instance are you viewing your profile on? Ie where is this screenshot from?

@KDederichs
Copy link
Author

It's my own one man instance (screenshot taken using Ivory).
You're right though, images used to work at some point:
Screenshot 2024-05-04 at 17 37 05
That's the one you posted a while ago.

So probably something broke between then and now

@KDederichs
Copy link
Author

Just FYI:
It also doesn't import new posts with images, I was digging through the logs to see if it actually processed stuff but nope:

nginx/access.log: [06/May/2024:20:03:16 +0000] "POST /inbox HTTP/2.0" 202 0 "-" "Bridgy Fed (https://fed.brid.gy/)"

journalctl:

May 06 20:03:16 mastodon bundle[1512218]: I, [2024-05-06T20:03:16.426814 #1512218]  INFO -- : [be8e8ebe-b0d4-4209-8128-ed72c59b6f32] method=POST path=/inbox format=*/* controller=ActivityPub::InboxesController action=create status=202 allocations=11672 duration=16.05 view=0.00 db=1.91 key=https://bsky.brid.gy/ap/did:plc:jaes5qxvg6ffn62rsqrrrtuc#key
May 06 20:03:16 mastodon bundle[1512185]: 2024-05-06T20:03:16.460Z pid=1512185 tid=2hc4ht class=DistributionWorker jid=8f788cdff35b85ed3e2e5e8c INFO: start
May 06 20:03:16 mastodon bundle[1512185]: 2024-05-06T20:03:16.461Z pid=1512185 tid=2fsnwd class=ActivityPub::ProcessingWorker jid=e0da406425e0e6556cfdbb5d elapsed=0.035 INFO: done
May 06 20:03:16 mastodon bundle[1512185]: 2024-05-06T20:03:16.483Z pid=1512185 tid=2fz06l class=FeedInsertWorker jid=5a7403d5069678bc96f5c1f4 INFO: start
May 06 20:03:16 mastodon bundle[1512185]: 2024-05-06T20:03:16.495Z pid=1512185 tid=2hc4ht class=DistributionWorker jid=8f788cdff35b85ed3e2e5e8c elapsed=0.035 INFO: done
May 06 20:03:16 mastodon bundle[1512185]: 2024-05-06T20:03:16.504Z pid=1512185 tid=2fz06l class=FeedInsertWorker jid=5a7403d5069678bc96f5c1f4 elapsed=0.021 INFO: done
May 06 20:03:28 mastodon bundle[1512185]: 2024-05-06T20:03:28.810Z pid=1512185 tid=2hh9ah class=LinkCrawlWorker jid=cf66c839d60ee8b9961498bf INFO: start
May 06 20:03:28 mastodon bundle[1512185]: 2024-05-06T20:03:28.815Z pid=1512185 tid=2hh9ah class=LinkCrawlWorker jid=cf66c839d60ee8b9961498bf elapsed=0.005 INFO: done
May 06 20:03:30 mastodon bundle[1512185]: 2024-05-06T20:03:30.262Z pid=1512185 tid=2fobkl class=ActivityPub::ProcessingWorker jid=1ec10e54264620be44a0336b INFO: start
May 06 20:03:30 mastodon bundle[1512210]: I, [2024-05-06T20:03:30.262545 #1512210]  INFO -- : [1695d8cf-7c25-4cb5-8767-3e164dafedfc] method=POST path=/inbox format=html controller=ActivityPub::InboxesController action=create status=202 allocations=11700 duration=16.06 view=0.00 db=1.77 key=https://infosec.exchange/users/briankrebs#main-key
May 06 20:03:30 mastodon bundle[1512185]: 2024-05-06T20:03:30.742Z pid=1512185 tid=2hh9ah class=ActivityPub::SynchronizeFeaturedCollectionWorker jid=b91c07d0f56081ea8a6c8d9e INFO: start
May 06 20:03:30 mastodon bundle[1512185]: 2024-05-06T20:03:30.744Z pid=1512185 tid=2hby35 class=ActivityPub::SynchronizeFeaturedTagsCollectionWorker jid=b7f58a733b3d07cd7089d7b5 INFO: start
May 06 20:03:30 mastodon bundle[1512185]: 2024-05-06T20:03:30.754Z pid=1512185 tid=2hc5e1 class=VerifyAccountLinksWorker jid=22e2deb05ad2bfb2d4d3d2a1 INFO: start
May 06 20:03:30 mastodon bundle[1512185]: 2024-05-06T20:03:30.823Z pid=1512185 tid=2fupup class=DistributionWorker jid=6498bece775d619e470d78ec INFO: start
May 06 20:03:30 mastodon bundle[1512185]: 2024-05-06T20:03:30.879Z pid=1512185 tid=2hby35 class=ActivityPub::SynchronizeFeaturedTagsCollectionWorker jid=b7f58a733b3d07cd7089d7b5 elapsed=0.135 INFO: done
May 06 20:03:30 mastodon bundle[1512185]: 2024-05-06T20:03:30.886Z pid=1512185 tid=2fupup class=DistributionWorker jid=6498bece775d619e470d78ec elapsed=0.063 INFO: done
May 06 20:03:30 mastodon bundle[1512185]: 2024-05-06T20:03:30.887Z pid=1512185 tid=2ht3cx class=DistributionWorker jid=02ceb210b0558a0ad72d2b2f INFO: start
May 06 20:03:30 mastodon bundle[1512185]: 2024-05-06T20:03:30.889Z pid=1512185 tid=2fobkl class=ActivityPub::ProcessingWorker jid=1ec10e54264620be44a0336b elapsed=0.627 INFO: done
May 06 20:03:30 mastodon bundle[1512185]: 2024-05-06T20:03:30.990Z pid=1512185 tid=2hiu1x class=FeedInsertWorker jid=d75e3978a9410a1d0ad8b3d0 INFO: start
May 06 20:03:30 mastodon bundle[1512185]: 2024-05-06T20:03:30.998Z pid=1512185 tid=2ht3cx class=DistributionWorker jid=02ceb210b0558a0ad72d2b2f elapsed=0.111 INFO: done
May 06 20:03:31 mastodon bundle[1512185]: 2024-05-06T20:03:31.018Z pid=1512185 tid=2hiu1x class=FeedInsertWorker jid=d75e3978a9410a1d0ad8b3d0 elapsed=0.028 INFO: done
May 06 20:03:31 mastodon bundle[1512185]: 2024-05-06T20:03:31.210Z pid=1512185 tid=2hh9ah class=ActivityPub::SynchronizeFeaturedCollectionWorker jid=b91c07d0f56081ea8a6c8d9e elapsed=0.468 INFO: done
May 06 20:03:34 mastodon bundle[1512185]: 2024-05-06T20:03:34.609Z pid=1512185 tid=2fejwp class=LinkCrawlWorker jid=aa47e6820bb8bc2a09a49488 INFO: start
May 06 20:03:34 mastodon bundle[1512185]: 2024-05-06T20:03:34.613Z pid=1512185 tid=2fejwp class=LinkCrawlWorker jid=aa47e6820bb8bc2a09a49488 elapsed=0.004 INFO: done
May 06 20:03:35 mastodon bundle[1512185]: 2024-05-06T20:03:35.665Z pid=1512185 tid=vzlx INFO: queueing Scheduler::IndexingScheduler (indexing_scheduler)
May 06 20:03:35 mastodon bundle[1512185]: 2024-05-06T20:03:35.665Z pid=1512185 tid=3gcmd INFO: queueing Scheduler::SuspendedUserCleanupScheduler (suspended_user_cleanup_scheduler)
May 06 20:03:35 mastodon bundle[1512185]: 2024-05-06T20:03:35.668Z pid=1512185 tid=2fevgt class=Scheduler::IndexingScheduler jid=dc84a213aff3f795e916486e INFO: start
May 06 20:03:35 mastodon bundle[1512185]: 2024-05-06T20:03:35.669Z pid=1512185 tid=2fyumh class=Scheduler::SuspendedUserCleanupScheduler jid=3cdf71d2ad5c7c119fd0a570 INFO: start
May 06 20:03:35 mastodon bundle[1512185]: 2024-05-06T20:03:35.807Z pid=1512185 tid=2fevgt class=Scheduler::IndexingScheduler jid=dc84a213aff3f795e916486e elapsed=0.139 INFO: done
May 06 20:03:37 mastodon bundle[1512185]: 2024-05-06T20:03:37.679Z pid=1512185 tid=2hc5e1 class=VerifyAccountLinksWorker jid=22e2deb05ad2bfb2d4d3d2a1 elapsed=6.925 INFO: done
May 06 20:03:43 mastodon bundle[1512185]: 2024-05-06T20:03:43.795Z pid=1512185 tid=2fyyyh class=LinkCrawlWorker jid=621bd9e4fd39b33b717e3af7 INFO: start
May 06 20:03:43 mastodon bundle[1512185]: 2024-05-06T20:03:43.799Z pid=1512185 tid=2fyyyh class=LinkCrawlWorker jid=621bd9e4fd39b33b717e3af7 elapsed=0.004 INFO: done
May 06 20:03:46 mastodon bundle[1512210]: I, [2024-05-06T20:03:46.106019 #1512210]  INFO -- : [85a256b4-2214-4a12-8956-15b6e08efa32] method=GET path=/api/v1/statuses/112395928317336783 format=*/* controller=Api::V1::StatusesController action=show status=200 allocations=15474 duration=34.93 view=17.96 db=5.78
May 06 20:03:46 mastodon bundle[1512218]: I, [2024-05-06T20:03:46.463740 #1512218]  INFO -- : [d66dafdb-46e9-463c-bb7e-9de6a3f834b3] method=GET path=/api/v1/notifications format=*/* controller=Api::V1::NotificationsController action=index status=200 allocations=3568 duration=11.84 view=0.41 db=2.92
May 06 20:03:46 mastodon bundle[1512210]: I, [2024-05-06T20:03:46.485220 #1512210]  INFO -- : [6b66f33a-0f15-41d2-b9b8-9f793a4afbe7] method=GET path=/api/v1/notifications format=*/* controller=Api::V1::NotificationsController action=index status=200 allocations=7697 duration=15.89 view=0.50 db=4.08
May 06 20:03:46 mastodon bundle[1512218]: I, [2024-05-06T20:03:46.599302 #1512218]  INFO -- : [e82ae822-320e-4bee-bcb7-83c95c26d9fb] method=GET path=/api/v1/timelines/home format=*/* controller=Api::V1::Timelines::HomeController action=show status=200 allocations=57077 duration=132.74 view=91.84 db=10.04
May 06 20:03:46 mastodon bundle[1512210]: I, [2024-05-06T20:03:46.700306 #1512210]  INFO -- : [48d95a4a-2203-4d6f-9200-726ce073ac1d] method=GET path=/api/v1/accounts/109298785969056377/statuses format=*/* controller=Api::V1::Accounts::StatusesController action=index status=200 allocations=194531 duration=247.42 view=164.09 db=9.76
May 06 20:04:01 mastodon bundle[1512185]: 2024-05-06T20:04:01.642Z pid=1512185 tid=2hvp75 class=ActivityPub::ProcessingWorker jid=8a926420a65a12204a99a96c INFO: start
May 06 20:04:01 mastodon bundle[1512218]: I, [2024-05-06T20:04:01.642481 #1512218]  INFO -- : [401fda6b-4129-40e6-b6ad-e5b20610c53d] method=POST path=/actor/inbox format=*/* controller=ActivityPub::InboxesController action=create status=202 allocations=17599 duration=56.60 view=0.00 db=1.12 key=https://relay.fedi.buzz/tag/furry#key
May 06 20:04:02 mastodon bundle[1512185]: 2024-05-06T20:04:02.149Z pid=1512185 tid=2fyxe5 class=LinkCrawlWorker jid=226e30051bc0166ad6b9bfe1 INFO: start
May 06 20:04:02 mastodon bundle[1512185]: 2024-05-06T20:04:02.743Z pid=1512185 tid=2fejwp class=ActivityPub::SynchronizeFeaturedCollectionWorker jid=ce96cb2d39dae04a442da830 INFO: start
May 06 20:04:02 mastodon bundle[1512185]: 2024-05-06T20:04:02.749Z pid=1512185 tid=2fupup class=ActivityPub::SynchronizeFeaturedTagsCollectionWorker jid=361b5bee4fc7ea482b367043 INFO: start
May 06 20:04:02 mastodon bundle[1512185]: 2024-05-06T20:04:02.759Z pid=1512185 tid=2fobkl class=VerifyAccountLinksWorker jid=284517dde588e0fdba092e07 INFO: start

As you can see there's no image processing of any kind going on here

@snarfed snarfed added the now label May 6, 2024
@snarfed snarfed added now and removed now labels May 9, 2024
snarfed added a commit to snarfed/dag-json that referenced this issue May 16, 2024
now supports AT Protocol's DAG-JSON variant with $link and $bytes: https://atproto.com/specs/data-model. for snarfed/bridgy-fed#1000
@snarfed snarfed closed this as completed May 16, 2024
@snarfed
Copy link
Owner

snarfed commented May 16, 2024

snarfed added a commit that referenced this issue May 16, 2024
…='atproto'

for AT Protocol's DAG-JSON variant with $link and $bytes: https://atproto.com/specs/data-model ; for #1000
@KDederichs
Copy link
Author

You sure it's fixed?
The account from above just posted 3 posts with images in them but now not even the posts are getting bridged.
Might be some unrelated new error though.

@snarfed
Copy link
Owner

snarfed commented May 19, 2024

Unrelated new error indeed. Catching up now, sorry!

@vyv03354
Copy link

vyv03354 commented May 19, 2024

Some Bluesky posts say that images are bridged only if they have ALT.
https://bsky.app/search?q=Bridgy+Fed+ALT

@vyv03354
Copy link

Example:
Original Bsky post => Bridged post: only an image with ALT is bridged.

@snarfed
Copy link
Owner

snarfed commented May 19, 2024

@vyv03354 hmm! Definitely a bug if true, not intentional. I'll look.

@snarfed snarfed reopened this May 19, 2024
@snarfed
Copy link
Owner

snarfed commented May 19, 2024

Confirmed, images with blank alt text aren't working. @vyv03354 thanks for the nudge!

snarfed added a commit to snarfed/granary that referenced this issue May 19, 2024
snarfed added a commit that referenced this issue May 19, 2024
…as objects

hopefully for #1000 (comment), ATProto images without alt text not showing up in AP
@snarfed
Copy link
Owner

snarfed commented May 19, 2024

@snarfed snarfed closed this as completed May 19, 2024
@pjreed
Copy link

pjreed commented Sep 22, 2024

@snarfed Although this issue was closed several months ago, I'm seeing it happen again.

Here's an example post where I don't see an image on my instance: https://fuzzyfox.social/notes/9yhco7nc0xat05we

But it does have an image on bsky: https://bsky.app/profile/did:plc:yqktid6kpzv52qfom6vgi752/post/3l4qyp5vnnm22

@snarfed
Copy link
Owner

snarfed commented Sep 22, 2024

@pjreed hmm! Interesting that the image isn't visible on fuzzyfox.social. I see it on other fediverse instances, eg indieweb.social, when I search for it by https://fed.brid.gy/r/https://bsky.app/profile/did:plc:yqktid6kpzv52qfom6vgi752/post/3l4qyp5vnnm22

image

@pjreed
Copy link

pjreed commented Sep 22, 2024

Interesting; it's possible this is somehow different. This is the most recent image I see in any post bridged from bsky, which does have alt text and is from two weeks ago:
https://fuzzyfox.social/notes/9y0fz03zdb2205uw

Doing a real quick skim over search results, I think the most recent image I see without alt text is from 3 months ago.

@pjreed
Copy link

pjreed commented Sep 22, 2024

I did a quick test from my own bsky account and can confirm that an image without alt text did not make it: https://fuzzyfox.social/notes/9yhfarnh0xat07x3

But one with alt text did: https://fuzzyfox.social/notes/9yhfbgro0xat07xj

@snarfed
Copy link
Owner

snarfed commented Sep 22, 2024

Interesting! Thanks for the testing. Sounds like Sharkey may be having interop trouble with Bridgy Fed.

Looking at the activities, the difference is that image is an object when there's alt text, and a bare string id when there isn't. Sharkey should handle both, but I suspect it's choking on the bare string id.

16,22c14,16
<   "image": {
<     "name": "An icon of a cartoon fox head in front of clouds",
<     "type": "Image",
<     "url": "https://porcini.us-east.host.bsky.network/xrpc/com.atproto.sync.getBlob?did=did:plc:i3jhyvg5pab4ac5ywqkpcuiz&cid=bafkreielmlhyb7rziwjxz4y2cvcunp2yta7wbalnmmg75dzwsquvzoiwuy"
<   },
--
>   "image": "https://porcini.us-east.host.bsky.network/xrpc/com.atproto.sync.getBlob?did=did:plc:i3jhyvg5pab4ac5ywqkpcuiz&cid=bafkreielmlhyb7rziwjxz4y2cvcunp2yta7wbalnmmg75dzwsquvzoiwuy",

without: https://fed.brid.gy/r/https://bsky.app/profile/did:plc:i3jhyvg5pab4ac5ywqkpcuiz/post/3l4r4siwwjm2b

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "id": "https://bsky.brid.gy/convert/ap/at://did:plc:i3jhyvg5pab4ac5ywqkpcuiz/app.bsky.feed.post/3l4r4siwwjm2b",
  "url": "https://fed.brid.gy/r/https://bsky.app/profile/did:plc:i3jhyvg5pab4ac5ywqkpcuiz/post/3l4r4siwwjm2b"
  "attributedTo": "https://bsky.brid.gy/ap/did:plc:i3jhyvg5pab4ac5ywqkpcuiz",
  "content": "<p>Testing bridging a post without alt text, ignore me!</p>",
  "contentMap": {"en": "Testing bridging a post without alt text, ignore me!"},
  "content_is_html": true,
  "attachment": [{
      "url": "https://porcini.us-east.host.bsky.network/xrpc/com.atproto.sync.getBlob?did=did:plc:i3jhyvg5pab4ac5ywqkpcuiz&cid=bafkreielmlhyb7rziwjxz4y2cvcunp2yta7wbalnmmg75dzwsquvzoiwuy"
    }],
  "image": "https://porcini.us-east.host.bsky.network/xrpc/com.atproto.sync.getBlob?did=did:plc:i3jhyvg5pab4ac5ywqkpcuiz&cid=bafkreielmlhyb7rziwjxz4y2cvcunp2yta7wbalnmmg75dzwsquvzoiwuy",
  "published": "2024-09-22T17:34:57.917Z",
  "to": ["https://www.w3.org/ns/activitystreams#Public"],
}

with: https://fed.brid.gy/r/https://bsky.app/profile/did:plc:i3jhyvg5pab4ac5ywqkpcuiz/post/3l4r4thygld2u

{
  "@context": "https://www.w3.org/ns/activitystreams",
  "type": "Note",
  "id": "https://bsky.brid.gy/convert/ap/at://did:plc:i3jhyvg5pab4ac5ywqkpcuiz/app.bsky.feed.post/3l4r4thygld2u",
  "url": "https://fed.brid.gy/r/https://bsky.app/profile/did:plc:i3jhyvg5pab4ac5ywqkpcuiz/post/3l4r4thygld2u"
  "attributedTo": "https://bsky.brid.gy/ap/did:plc:i3jhyvg5pab4ac5ywqkpcuiz",
  "content": "<p>Testing bridging a post with alt text, ignore me!</p>",
  "contentMap": {
    "en": "Testing bridging a post with alt text, ignore me!"
  },
  "content_is_html": true,
  "attachment": [{
      "name": "An icon of a cartoon fox head in front of clouds",
      "type": "Image",
      "url": "https://porcini.us-east.host.bsky.network/xrpc/com.atproto.sync.getBlob?did=did:plc:i3jhyvg5pab4ac5ywqkpcuiz&cid=bafkreielmlhyb7rziwjxz4y2cvcunp2yta7wbalnmmg75dzwsquvzoiwuy"
    }],
  "image": {
    "name": "An icon of a cartoon fox head in front of clouds",
    "type": "Image",
    "url": "https://porcini.us-east.host.bsky.network/xrpc/com.atproto.sync.getBlob?did=did:plc:i3jhyvg5pab4ac5ywqkpcuiz&cid=bafkreielmlhyb7rziwjxz4y2cvcunp2yta7wbalnmmg75dzwsquvzoiwuy"
  },
  "published": "2024-09-22T17:35:30.468Z",
  "to": ["https://www.w3.org/ns/activitystreams#Public"],
}

@snarfed snarfed reopened this Sep 22, 2024
@pjreed
Copy link

pjreed commented Sep 22, 2024

Out of curiosity, do you have a way to test whether this is Sharkey-specific or also an issue with upstream Misskey? I don't have an account on a Misskey server I can easily test with...

@snarfed
Copy link
Owner

snarfed commented Sep 22, 2024

I'd have to test in prod like you, I don't have local *key instances that I test against. Fortunately if both are broken, fixing it on BF's side will likely fix both.

@derspyy
Copy link

derspyy commented Sep 30, 2024

mastodon seems to just use the attachment field, where name is null in the absence of alt-text.

image is unused here.

"attachment":[
{
"type":"Document",
"mediaType":"image/png",
"url":"https://files.mastodon.social/media_attachments/files/113/229/002/117/467/328/original/5adf8e3cfc7e9430.png",
"name":null,
"blurhash":"U3D,Dn}_=~%L%3NYxbt7FW-X$lRj%NjJI-s;",
"width":1055,
"height":880
}
],

source: https://activitypub.academy/activitypub_explorer

@derspyy
Copy link

derspyy commented Sep 30, 2024

seems like we need at least type on the attachment (mastodon defaults it to Document which is probably a good placeholder even if you want to handle it properly at some point).

@snarfed
Copy link
Owner

snarfed commented Oct 2, 2024

I generally aim at simple AS2 that interops as well as possible, as opposed to "whatever Mastodon does." That way lie dragons. 😁 Actual standards are better!

For this, I've made BF always emit images as objects with type: Image, and not as bare string URLs any more. Example: https://fed.brid.gy/r/https://bsky.app/profile/tbray.org/post/3l5ksdm4t7223 . @KDederichs feel free to try again!

@derspyy
Copy link

derspyy commented Oct 3, 2024

seems like images are arriving @ capivarinha.club 😄 https://capivarinha.club/notes/9yw6q32fqqa12ar9

@snarfed
Copy link
Owner

snarfed commented Oct 3, 2024

Thanks for confirming @derspyy! Tentatively closing. @pjreed feel free to reopen if you still aren't seeing images in new posts!

@snarfed snarfed closed this as completed Oct 3, 2024
@pjreed
Copy link

pjreed commented Oct 4, 2024

Thanks, I just checked and it looks good to me now!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

5 participants