Justin (admin of Fedisearch) @fedisearch@qoto.org

@freemo
True true. Adding some insights on behind the scenes, our crawler only visits mastodon, pleroma and misskey instances, because it only knows how to parse content from those instances. Some hubzilla posts get indexed because they appeared on the federated timelines of those three type of instances.

@hansw @fedisearch_com

@freemo

Yup this is indeed the way we are aiming for. However, still facing some technical difficulties with the performance when applying this method on mega threads where many people are being mentioned.

@hansw regarding indexing of @ mentions, methods to achieve it is still being explored.

Does the opting out section in https://fedisearch.com/about help?

@rgx@muensterland.social thank you :)

With the help from @8zu 🎉 , we recently localised https://fedisearch.com in Chinese and Japanese

你好！中文用户

こんにちは！日本のユーザー

@hansw @Gargron @dansup

I get your point indeed, and we take users' privacy very very seriously. I think the main problem here is that Pixelfed doesn't support the semantic to allow users to opt out from search indexing. In the case of mastodon, we support this feature.

The same set of settings that blocks google from indexing you would also block fedisearch from doing so as well. We have a insignificant amount of traffic compared to google... so if being indexed is the main worry, we feel we shouldn't be the main antagonist 😅

@hansw @dansup I think it would really be up to fediverse softwares (in our case Pixelfed) to support noindex and noarchive metatag in order to be excluded from indexing. Mastodon already has this feature

@namark

I'll leave the link to Toot! https://apps.apple.com/us/app/toot/id1229021451 a well received iOS app for mastodon, commercial and proprietary, yet makes 4.5/5 stars on 132 ratings.

And, publicly endorsed by mastodon https://joinmastodon.org/apps

Withdrawing from further discussions

@hansw I understand the concern, and personally hate spammers a lot too.

We try to strike for a balance between out of the box utility and respects for people's desire to stay out of the spotlight, and it indeed has been hard.

If we search pixelfed + a username on Google, we'd find many results. It wouldn't be fair for Google to grab all the search results without opt in while a smaller niche focused projects has to knock on doors right?

Not saying that defaulting to opt-out is right, but just want some extra leniency on our way to do something good to this community.

@namark I don't see mastodon and other community publically deny access to the network from close sourced software. Would you want to substantiate?

The use of analytic solutions on fedisearch is fully disclosed in our privacy policy; data collected are not personally identifiable.

@hansw for sure. Looks like we need to implement self serve immeidate opt out sooner

@hansw I think this is very valid concern. Sorry we let this one slip through. We will resolve this.

@hansw if you are talking about license in the context of open-source licenses (GPL, BSD for sample), they only apply to open-source softwares. In our case, no sourcecode license is granted

@hansw your fediverse software should support this feature, just like mastodon and pleroma do. If it doesn't, it would be wise to ask your admin to add this feature