for what it's worth i grepped through my webserver's access logs for my static blog out of curiosity earlier today due to this particular 'discourse'

mastodon useragent had just under 40k entries while useragents for other fedi software were nonzero but none was above 50 entries and combined probably about 100 entries total. out of 50k total entries. there's definitely something mastodon is doing differently than other fedi software when it comes to how they treat external links. mind you, my situation is mostly link verification rather than previews but it seems to be similar behavior

either that, or all the other fedi software is using some different useragents that don't line up with the name of the software.. but i kinda put out an ask about that privately earlier today if that's the case for other fedi software (or if other fedi software uses mastodon useragent for external crawling for some reason) and didn't really get anyone telling me anything that violated my assumptions

@rudi AFAIK that's more entries than there are (known) Mastodon instances. Is that 40k within less than two weeks, accessing the same URL? Definitely strange if so.

@box464 looked at the preview card generation behavior last year (box464.com/posts/mastodon-prev…) and conjectured that a page may be accessed repeatedly if it has OpenGraph tags, but not an OpenGraph image. That doesn't appear to apply to your blog though.

@gabek @jwz

Understanding Mastodon Preview Card Display Logic

Ever wondered why some links you post to Mastodon show a nicely formatted preview card, but others don’t? It can seem random, even showing differently from app to app.

^{Jeff Sikes}