I’ve thought about this when @ar.al🌻 shared their massive #Mastodon worker activity increase related to the #TwitterMigration. This migration barely started, relative to the total number of Twitter accounts, and the #Fediverse wasn’t particularly under-provisioned.
It’s just that activity increase isn’t linear with the total number of users/instances. And as long as we were a marginal item, existing hosting solutions could stay ahead of the curve.
The Twitter migration isn’t the real deal, it’s a relatively small stress test, but its effects already are hard to handle for most existing servers.
I believe we may be at a turning point where no #Fediverse project will be able to claim to be “light” or “fast” anymore, just by virtue of the compounded network load.
It will be interesting to watch live how Fediverse scales up in reality.Thing is: Fediverse’s communication between instances is roughly quadratic in the number of instances, and it’s getting closer to quadratic as the instances are becoming more interconnected due to increased user numbers.
An additional issue: the effort of running an instance (e.g. storage space) isn’t linear in the number of instance users but rather roughly linear in the number of total Fediverse users across all instances.
In other words: if at some point the majority of Twitter users migrates to Mastodon or other Fediverse software, spreading the load across a huge number of instances won’t make it all too much easier to handle. On the other hand, the traffic required to keep all these instances in sync might become completely unrealistic.
Interesting times…
Max Kostikov likes this.
reshared this
Ed Chivers
in reply to Hypolite Petovan • • •I was wondering about this yesterday - I am a newbie here but I've noticed that the instance I'm on gets pretty slow during the North American day. With all these instances talking to each other, it feels like linear user growth is going to ramp up traffic by some kind of square (or exponential?) factor. I don't know how the system is designed, whether it's many-to-many or whether some sort of hub-and-spoke design might help?
But, as I said, total newbie, just thinking out loud.
Hypolite Petovan
in reply to Ed Chivers • • •@Ed Chivers Like @infosec.exchange/users/WPalant mentioned in their posts I linked, it seems to be quadratic to the number of instances and the efforts to run a single server is linear with the total number of Fediverse users which sounds reasonably accurate to me.
It is many to many, even accounting for the server blocks, and I'm not sure how a hub-and-spoke model would work given the underlying protocol design decisions.
Ed Chivers
in reply to Hypolite Petovan • • •Hypolite Petovan
in reply to Ed Chivers • • •Darnell Clayton
in reply to Hypolite Petovan • • •One possible solution is to aggressively advocate for most hosting companies to take Federated software more seriously.
Many are nervous about hosting social networks due to copyright issues, but the appeal of new customers might make it reasonable for them to consider.
Hypolite Petovan likes this.
Becky
in reply to Hypolite Petovan • •Hypolite Petovan
in reply to Becky • • •Becky likes this.
Becky
in reply to Hypolite Petovan • •Ah, I've actually read it now 😓
I hope we can figure it out
Hypolite Petovan likes this.
Hypolite Petovan
in reply to Becky • • •Becky likes this.
Yellow Flag
in reply to Becky • • •@firefly_lightning No, that’s exactly what is going to make things worse as mentioned in my post. More instances means more communication between instances. And this grows quadratically. It will reach the point where no connection will be wide enough to handle the traffic way too soon.
@hypolite
Hypolite Petovan likes this.
clacke: exhausted pixie dream boy 🇸🇪🇭🇰💙💛
in reply to Hypolite Petovan • • •If I'm a single-user instance and follow 1000 people, it doesn't matter to me if the rest of fedi is 100 k people or 1 M people, I'll have the same traffic ... unless they all start following me.
I guess a bigger network means followers, but I don't see how it's superlinear. Especially my load doesn't change if 1 person or 1000 people follow me from mastodon.online. And certainly not my storage space.
@Hypolite Petovan @ar.al🌻 @Yellow Flag
Hypolite Petovan
in reply to clacke: exhausted pixie dream boy 🇸🇪🇭🇰💙💛 • • •Hypolite Petovan
in reply to Hypolite Petovan • • •@a Claes unto himself 🇸🇪🇭🇰💙💛 Case in point, my own single-user node is aware of 12,406 remote instances, and I have only 1,000 contacts myself. But other people like and reply to my posts, so they are probed as well. So my node maintains records for 216,392 contacts.
Most of these contacts would be shared in larger instances, but single-user instances need to keep tabs of remote contacts independently, increasing the network traffic.
Hypolite Petovan
in reply to Hypolite Petovan • • •Hypolite Petovan
Unknown parent • • •