Ease of use, open source, and SaaS
In a recent project I was setting up a Postgres database. I've done this many times over many projects. None of it is a surprise. Making it resilient to errors and data loss still is however.
Even though I'd set up a production Postgres cluster a decade ago (with read replicas and continuous archiving such that even when I accidentally deleted the production database there was no data loss and a few minutes downtime) I caught myself off guard.
Surprise! Data loss. On my own personal project.
How is that possible? Not "how was I an idiot this time" (as I promise you there will never be a lack of idiocy from me) but simply "how wasn't I saved by the tooling?"...
I'm certainly no genius at databases but I have actual production experience and this still caught me out. And having a safe place to put your data such that it won't disappear is a foundational bedrock of modern computing.
The incident itself
docker compose up starts all containers. docker compose down removes containers - without asking. What I had intended to do was docker compose stop.
The Docker Postgres defaults aren't trivial. I did miss a key core default.
Backup script with cron job just outside of the daily backup window as continuous archiving is not trivial.
Deferred or ignored more resilient backup solutions as it's not trivial and the worst was some personal data loss.
The broader question
Open source and developer friendly
Good user interfaces, whether that means trivial to set up or resilient to user error, require careful work and design. Such a user interface generally also must be made by an expert with the specific intent on assisting and playing out the experience of a new user. Most of the time it also requires sacrificing flexibility and customizability (see: skill floor vs skill ceiling) to make the Pareto optimal workflow fully supported and debugged.
Self serve software creation and extension, as seen in open source, generally stands counter to that.
Open source is unpaid for the most part. Most projects are crafted to directly serve the creator's problems. The initial and default user for a tool is the creator themselves, then perhaps a few die-hard (and technical) users, and so on. This is brilliant. Worrying about everyone else, the general public, is an easy way to lose momentum and sanity - especially when you're not being paid for the inevitable deluge of customer support.
SaaS as both the solution and problem
Imagine you wanted to set up a database. It's an open source database. The majority of the tooling for reliably deploying and maintaining this database exist as open source. Yet most developers will suggest and/or default to paying for SaaS. Why is that?
Maintaining and backing up a database is not necessarily "difficult" but it can be complicated, and where there's complication there's potential error. When an error might lead to data loss an error prone approach is generally no longer deemed an option.
SaaS not only own the hardware but their proprietary tooling on top of open source libraries can make an unreliable and error prone process into a reliable one.
As we lean on SaaS overarching layers to provide the seamless experience however the underlying open source layers and process become less and less well exercised. The experience of using the open source database with and without SaaS may no longer even be comparable. Those installing and running their own system, away from the SaaS seamless experience, likely have very particular issues or needs. The default "new user" for the open source project is now far and away from the novice user.
We now have a funnel of money that's no longer going to the open source process to deal with the most complicated piece of the puzzle for new users: ease of use.