Every time I go to the piefed frontpage I’m blown away by how much more polished it is. It has all the bells and whistles that lemmy is sometimes missing.
Whats the catch? Why aren’t we recommending everyone goes to piefed instead of lemmy?
App support is one thing I can think of.
We have data on what it costs to run a sizeable instance of Lemmy and it’s not a lot. How does Piefed compare? Anyone starting an instance who envisions it growing large has to contend with this question. Currently it seems it’s got a bit under 1000 users across under 10 servers.
There are now sizeable communities run on Lemmy instances that are reinforced by network effects. There needs to be a significant reason for them to migrate. To that point, the collective project is building communities away from corporate power, not software. The software is a tool to facilitate that. Lemmy has worked well so far in this regard. If someone can show that Piefed can work better and not cost significantly more, it’ll probably get adopted for new communities. If the difference is drastic, we may even see migrations from Lemmy.
We won’t 100% know the answer to that until we get there. But in 2025 fear of a lack of CPU cores is NOT what keeps me awake at night.
Early performance results are positive. Check these links out:
https://join.piefed.social/2024/02/13/technical-performance-of-each-fediverse-platform/
https://join.piefed.social/2024/02/09/comparing-network-utilization-of-lemmy-kbin-and-piefed/
There are many many ways to ruin web app performance and choice of backend language is not really a big one. It’s what you do with it that counts.
https://piefed.social/ is running on a low end VPS which costs $7.50 per month. Load average is about 1.45 during the busiest part of the day. Most of the load is caused by federating with lemmy.world and that won’t increase as more users come on board.
PieFed is also really efficient with storage. After 16 months of operation, subscribed to every popular community, the piefed.social DB is 30 GB and the media storage is 28 GB. A Lemmy instance would be 10x that. I haven’t bothered to add S3 storage code because we just don’t need it (yet).
Anyway, all this focus on costs and downsides is only half the coin. There are massive benefits that come from using Python:
For a FOSS project where volunteer contributions from people play a big part these things are really important. There are many ways a project can fail (not just technical reasons but social & governance too) and running out of CPU is way way down on the list.
These performance results are only from the browser side, but dont cover server performance. The database for lemmy.ml is 60 GB, and that is with 6 years of history. Not sure where your 10x claim comes from. The lemmy.ml server costs 70 Euros per month and doesnt have much loa, with almost 10 times as many active users.
I know the 0.03€ per user per month has been known for a while, but it still impresses me.
Full disclosure: I like Python a lot and have written a lot of it.
That said, if not for my recent work experiences, I would be absolutely horrified at the idea of using Python for such a project. Between the type system and being interpreted, the performance and runtime issues are pretty painful. That and the historical greater dependence on external application servers really makes Python-based services something that really sucks to administer.
However, as I noted, I have also recently seen Python performing far faster than it has any right to with highly-optimized use of multi-processing and offloading the server stuff to Go.
I think I’m going to have to take a look at Piefed source this weekend.
Cool! Before you dive in, check this out https://join.piefed.social/docs/developers/
That’s nice info, thanks! Do you know why the db difference?
I’ve never seen a Lemmy DB, sorry. But I hang out in the Lemmy matrix rooms and read about admins struggling with their 300 GB databases quite often.
Can confirm. When we took over the running of feddit.uk migrating the images took forever as it was around 300GB.
Migrating the images as in media? The discussion is about database sizes.
The biggest DB I have is the one from alien.top, which got close to deal with 600k mirrored bots and 10M posts + comments. The database was clocking around 25GB.
I second this. Lemmy is written in Rust where as piefed is written in Python. When it comes to running a high-performance webserver, Lemmy has the advantage.
While theoretically true, the main bottleneck with Lemmy seems to be the database performance, so with both projects depending on PostgreSQL for that, I somewhat doubt that Piefed being written in Python will have much noticeable effect in reality.
Postgres being a bottleneck is a first for me. Not saying it’s not possible, just… It’s postgres. Wondering if it’s more an issue with ORM, etc.
Postgres is so quick if you know how to use it…
You don’t even need to know how to use it very well, in my experience.
Really depends on many factors. If you have everything in RAM, almost nothing matters.
If your dataset outgrows the capacity, various things start to matter, based on your workload. Random reads need to have good indices (also writes with unique columns), OLAPs benefit from work_mem, >100M rows will need good partitioning, OLTP may even need some custom solutions if you need to keep a long history, but not for every transaction.
But even with >B of rows, Postgres can handle it with relative ease, if you know what you’re doing. Usually even on a hardware you would consider absolutely inadequate (last year I migrated our company DB from MySQL to Postgres, and with even more data and more complex workflows we downsized our RAM by more than half).
Yeah, this would be my concern as well if I had to run it. Sure Python apps can be fast and most time is spent in IO, not compute, and if you’re running a profitable operation the exact cost of compute might not matter much. However if you’re running a non-profit service and you want it to be as dirt cheap as possible so it can be free for most users, then the cost of compute very much does matter.
If you want it to be “free to most users”, the cost of data storage and IO will completely dominate over the cost of CPU.
There are plenty of good arguments to prefer Rust over python for a distributed application, but “language efficiency” is not one of them.
Anyway, if you are biased in favor of Rust and want a decent argument to justify it, I will let you use ‘It’s easier to compile Rust to WASM and have the application run on the browser, while compiling python in a cross-platform way is a nightmare’, free of charge.
Ease of cross-compiling is really one of my favorite things about Rust. It can run anywhere with little coaxing needed.
I don’t think this is a major concern yet. The largest PieFed instance has 308 active users, 2nd place has 34. They’ve got room to grow.
https://piefed.fediverse.observer/list
People can start posting about PieFed on Reddit and see how the Reddit users react.
But how is that not a concern if you’re interested in attracting more users? You run an instance with 500 users. Some thread on Reddit explodes and you get 1000-10000 new users in a few days. If Piefed has poor scaling you might be unable to pay the bills for your now much larger instance. That’s not gonna be great for you or the new users.
I think it’s unlikely that they would attract such a large number of users with 1 post on r/RedditAlternatives or something. Lemmy gets spammed everywhere and we usually don’t even gain 1000 users a day overall across all instances.
There’s already been some comments about PieFed and they didn’t result in huge surges.
Sure but does the rate of growth matter? The post asks about recommending Piefed instead of Lemmy. I presume the point is that the number of Piefed users would grow if we did that. So whether a thread produces 10, 1000, or 10000 users in a day, the number of users would grow over time. Then I think the question remains, if my Piefed instance costs $10/mo to run today, would it cost $100 with 10000 users or $1000, or more, or less?
Relevant
The rate of growth does matter yea. If an instance gets worried, they can lock signups. Slow growth means the software has time to improve as they notice issues.
Lemmy had many issues scaling before, except Lemmy had huge surges with the Reddit API blackouts.
If people start recommending PieFed now, it’s on their own terms instead of a massive wave. They can backoff if they get too many users.
Yeah, that makes sense for the defect class of performance problems. I’m more concerned with the inherent performance (compute) disadvantages of Python. Perhaps they wouldn’t matter, hard to know without load testing.
I didn’t downvote.
Well, i thought the same about rust and python. i’m not a dev and lack understanding on their component.
My opinion became more nuanced with Lemmy. I have read there spike in ressource usage with lemmy on server.
So, I believe the language matter but also its features, the way you think how people will use it. Its core design.
When you manage a social network, the most fondamental feature is the moderation board. I find this area is lacking on Lemmy. Crosspost ? Duplicate everywhere, on my timeline, i get 4-5 time the same post…so i believe it miss some design thinking.
From what i see, the various lemmy-bot (most of them are written in python i think ?) and alternative frontend are improving lemmy experience. For example, how do you add an mod with lemmy-web-ui ?
I also believe that being written in Rust make it harder for new devs to help. But in later time we will see a change.
I’m not sure if it answer part of your question, but there are area where lemmy can improve : https://join.piefed.social/2024/02/09/comparing-network-utilization-of-lemmy-kbin-and-piefed/
Its written in python, but I don’t think the overhead is too much because the bottleneck is DB performance.
It has support for lemmy’s protocol, so the network effect really isn’t an issue.
Small lapsus ha ha
Thanks.