@thisismissem @Gargron @jonny @onepict

The problem, as Gargron identified, appears to be the lack of a "noindex" tag, which in Fediverse terms is like running an SMTP open relay - a misconfiguration rather than a fault in protocol - but which should not be the default in any software, and which will get you instablocked by the entire Internet.

@dentangle @Gargron @jonny @onepict right, best practice is to not make remote content directly viewable without authentication (but it may still appear in thread/reply views without authentication)

@thisismissem @Gargron @jonny @onepict yes, where "best practice" == "if I don't want my instance defederated by the majority of the fediverse"

@dentangle @Gargron @jonny @onepict the source of that best practice is more around rehosting random content and consequently having liability for that content.

@thisismissem @Gargron @jonny @onepict

That may be the case for some instance admins, but most users are not admins.

The bigger issue is that feeding fediverse toots into search engines violates conventions and the expectations of most users. That's what causes fedi-riots every time some bright spark does it.

Hi @julian

I know you're very busy sitting on panels at #Fedicon and talking about how to make the fediverse better. Great.

Unfortunately you are still running a scraper that is feeding search engines.

You've been posting from the con (tip: we use alt text on pictures here on the fediverse), so I know you're online.

You're following me, so you'll have seen my question. @Gargron has spoken to you too I believe.

A day later, no acknowledgement or apology or fix or promise of a fix. Why?

Hi dentangle@chaos.social, I haven't been at a laptop this entire day since 7am this morning.

Around then I added a change to the link tags sent for remote profiles so that they point to the canonical source (your actual profile).

I'll likely just put in a redirect to your profile so it won't be accessible.

dentangle@chaos.social I appreciate your civility so far while I work through what needs to be done about this.

@dentangle@chaos.social Quick question, what makes you think this is a scraper? NodeBB is forum software that implements ActivityPub and federates using the protocol.

@deadsuperhero It doesn't matter where the data is coming from, the effect is the same. Scraping done over AP is still scraping. The data (retrieved over AP in this case) is being republished without a "noindex" tag so it is being fed into search engines, including posts on your peertube server.

@julian Thank you for your response and taking this seriously.

Please keep everyone informed. Feeding fediverse data to search engines (even accidentally, as this appears to be) is a breach of trust. How you handle this now is likely to be remembered by the fediverse for a long time.

dentangle@chaos.social the noindex tag has been added to all remote profiles.