Go Back   Cockos Incorporated Forums > REAPER Forums > REAPER for Linux

Reply
 
Thread Tools Display Modes
Old 11-23-2021, 10:53 AM   #1
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default Fascinating Reading: Proprietary Audio Plugins for Linux, but the right way

I just discovered and read this fascinating thread by well known Linux developers from Bitwig, yabridge, Ardour, Surge, Overtone, etc. For anyone wanting to better understand the state of proprietary audio plugins currently on Linux, as well as better understanding the difficulties these developers go through to provide these plugins (versus Windows or Mac), this thread won't disappoint!! There were so many takeaways from this conversation! It was really, REALLY eye opening! :-)

* It makes so much more sense why the developers seem to only put out proprietary packages for just Ubuntu (and sometimes Debian).

* It shows how much work goes into creating and supporting Linux binaries.

* It gives me a huge insight on how small the market share is currently for Linux, and yet also how it is growing daily.

* It shows the collaborative nature of these awesome Linux developers.

* And especially, I understand much better the strategies and reasonings for plugin library management, reducing the number of dependencies, etc. I don't think I will ever complain about ugly, sparse Linux GUIs ever again! :-)

* I understand why Chris (of AirWindows fame) uses no GUIs.

I understand why Reaper is so minimalist and why it works as well as it does.

There are these and many more takeaways in this fascinating thread:

https://gist.github.com/abique/4c1b9...591d2a7c760db4

Last edited by audiojunkie; 11-23-2021 at 12:59 PM. Reason: To eliminate evidence of poor spelling. ;-)
audiojunkie is online now   Reply With Quote
Old 11-23-2021, 11:01 AM   #2
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default

In addition, to supplement the Gist thread above, I found the following article a very useful and informative addition:

https://en.wikipedia.org/wiki/Dependency_hell
audiojunkie is online now   Reply With Quote
Old 11-23-2021, 11:22 AM   #3
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default

One thing that I want to research more is Arch/Debian pros/cons: Arch User Repository vs Ubuntu native distro supported proprietary plugins. Both of these distro families each have a unique strength, and I would LOVE to find a way to have both, in the most compatible way possible. For example, the number of unique source packages in the AUR far outweighs any other "common" distro family. And yet, Ubuntu seems to be the binary of choice for proprietary plugin developers. I guess my future recommendations for choice of audio distro will be to ask the querying user where they plan to get the majority of their audio software? ie Open Source or Proprietary purchases.

If the majority of the plugins and software that a user will be utilizing will come from open source, then the Arch family would be the best recommendation, because of the AUR.

However, if the majority of the plugins and software that a user will but utilizing will come from proprietary software (ie freeware or for purchase), my future recommendation may point to the Debian family, and in particular, Ubuntu.

Personally, I wish this would change, because there are so many reasons that I don't want to use Ubuntu, and currently prefer Arch. I'll need to assess things again. As it stands, of the two families, my personal preferences would be Arch or Debian over Ubuntu, because of the freedom and choice I get from independent over corporate controlled distributions.

So, as it stands, as long as the KXStudio repos remain well maintained and up to date, it looks like Ubuntu may have the edge because developers prioritize Ubuntu for proprietary audio plugin development and because the KXStudio repos provide most of what the AUR would provide as far as audio software goes. However, if the KXStudio Repos are not well maintained, reliable and kept up to date, the leading edge goes to Arch for their Arch User Repository support.

What would be nice would be if developers support at least Ubuntu and Arch for proprietary binaries, even of they don't support other distros. Fedora, OpenSuse and Gentoo all have audio packages too, but because of the incompleteness of their repositories, they would be less than optimal targets for proprietary binary support, because more people will likely use either AUR supported distros or Ubuntu for their DAWs.

....which brings us all back around to the developer's discussions of the best way of supporting Proprietary Audio Plugins for multiple Linux distro families.

Last edited by audiojunkie; 11-23-2021 at 11:56 AM.
audiojunkie is online now   Reply With Quote
Old 11-23-2021, 11:37 AM   #4
flechtwerk
Human being with feelings
 
Join Date: Oct 2020
Posts: 32
Default

Quote:
Originally Posted by audiojunkie View Post
I just discovered and read this fascinating thread by well known Linux developers from Bitwig, Ardour, Ardour, Surge, Overtone, etc. For anyone wanting to better understand the state of proprietary audio plugins currently on Linux, as well as better understanding the difficulties these developers go through to provide these plugins (versus Windows or Mac), this thread won't disappoint!! There were so many takeaways from this conversation! It was really, REALLY eye opening! :-)
Oh wow, thanks for the link! I'll dive in straight away.
flechtwerk is online now   Reply With Quote
Old 11-23-2021, 01:01 PM   #5
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default

By the way, @robbert-vdh I accidently wrote Ardour twice when I meant to add your name for yabridge! No slight was intended, and the original post has been corrected. Sorry!
audiojunkie is online now   Reply With Quote
Old 11-23-2021, 10:36 PM   #6
4duhwinnn
Human being with feelings
 
Join Date: Mar 2017
Posts: 670
Default

Thanks for sharing the results of your ongoing research!
I think one of the troubles that insures a small linux userbase, is that so many distro maintainers create or use half-baked system installation routines, followed by inflexible login managers, followed by sundry new-and-improved package management systems, making it needlessly difficult for new linux users to test and to get up and running with the latest distros, which ideally would be easily populated with current software releases, and in a system gui chosen from a login manager that allows the user to choose among the installed possibilities at every startup cycle.

On the bright side, I doubt Linux 11 will be released any time soon...
Cheers
4duhwinnn is offline   Reply With Quote
Old 11-24-2021, 07:35 AM   #7
shosty
Human being with feelings
 
Join Date: Aug 2015
Posts: 240
Default

The kxstudio repos are not well maintained at the moment for anything other than the kxstudio apps. I think plugins have taken a backseat as falktx appears to be working hard on other things rather than spending time packaging.

It makes me keen to move to arch. However, I did install manjaro on a laptop and installed loads of stuff with AUR which was great but some plugins were not recognised as the same by Reaper and Renoise. Perhaps there is a simple fix for that by editing the project files but it's an issue which probably will keep me on KDE Neon for a while.
shosty is offline   Reply With Quote
Old 11-24-2021, 08:59 AM   #8
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default

Quote:
Originally Posted by shosty View Post
The kxstudio repos are not well maintained at the moment for anything other than the kxstudio apps. I think plugins have taken a backseat as falktx appears to be working hard on other things rather than spending time packaging.

It makes me keen to move to arch. However, I did install manjaro on a laptop and installed loads of stuff with AUR which was great but some plugins were not recognised as the same by Reaper and Renoise. Perhaps there is a simple fix for that by editing the project files but it's an issue which probably will keep me on KDE Neon for a while.
Very good to know. This has been an on-going concern for me. Without KXStudio and commercial binaries, the Debian family (as much as I personally love Debian), is no better than Fedora or OpenSuse (or any other non-AUR using distro). If only we could get the developers to provide binaries for commercial apps for the Arch family (instead of everything just being Ubuntu), the Pro Audio distro question would be resolved. With the Arch family, there is something for everyone: Manjaro is as easy as Ubuntu. Arch is for those who want to do things their own way. And EndeavourOS is for Intermediate users. Combined with the AUR, it's a very powerful distro family.

The KXStudio repositories are a tremendous resource, but it is kept and maintained by only one superhuman. The Arch User Repository, however, is kept and maintained by hundreds of experienced packagers. There are better odds of the AUR not going away than there are of the KXStudios no going away. In fact, IIRC, the KXStudio Repos did go away at one time for some issue. It was only a couple of months, but there's still the issue of the KXStudio repos being kept up and maintained well. I just personally think the Arch family would be better than Ubuntu for primary commercial support. Thank goodness Linux developers are working hard to try to maintain as broad of support as possible (including the Arch family).
audiojunkie is online now   Reply With Quote
Old 11-24-2021, 09:56 AM   #9
shosty
Human being with feelings
 
Join Date: Aug 2015
Posts: 240
Default

Is there much of a problem with commercial not working on Arch except for it coming in .deb form?
shosty is offline   Reply With Quote
Old 11-24-2021, 11:48 AM   #10
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default

Quote:
Originally Posted by shosty View Post
Is there much of a problem with commercial not working on Arch except for it coming in .deb form?
Most of the developers will say something in the requirements like:

"Works with Ubuntu 18.04 or higher."

Some will say:

"Works with Ubuntu 18.04 or similar."

The second may be more flexible, as it may indicate that a "similar" distro may also work, but it's always best to check with the developer first.

The key thing to remember though, is that when a developer compiles a project in a VM with a particular distro, and then tests it for that particular distro, he is only truly supporting that distro that he compiled and tested on. It's just like how Reaper is labeled, "Experimental" as a way of limiting the business liability.

When it comes right down to it, these developers are actively trying to support the broadest number of distros as possible. And, more than likely, the plugins will work on different distros (because the developers purposely try to program the code to be as compatible as possible. However, there are no promises. They've stated that they have built and tested and support Ubuntu Ubuntu 18.04 and up. It gives them legal room and it lets the customer know ahead of time what they are buying.

Like I said, it will likely work. But, because of the differences between different versions, there are risks of it not working. See:

https://en.wikipedia.org/wiki/Dependency_hell

Often what developers will do, is provide a demo version, which you are expected to try on your machine to make sure the app will work, before purchasing. This way, you can know if it will likely work or not.
audiojunkie is online now   Reply With Quote
Old 11-24-2021, 01:19 PM   #11
shosty
Human being with feelings
 
Join Date: Aug 2015
Posts: 240
Default

I really know next to nothing about developing but I think it's good that they use that as a base to be honest. The flip side is that if they develop for Arch then it's quite possible that only people on bleeding edge will be able to use them and I think that number is less than those using more conservative distros. Most linux savvy devs I've seen (VCV Rack being a good example) use an old toolchain to build against because if it works on old versions of packages/libraries/compilers then most of the time it will work with new versions too. The reverse is much more of a gamble and for binaries it can result in them not running at all.

The problems with running more bleeding edge distros aren't confined to closed source stuff though, I don't know if it's still an issue but there used to be VST versions of the gxplugins which wouldn't show their UI in hosts when using Arch because of a fundamental library version difference (I don't remember which it was now).
shosty is offline   Reply With Quote
Old 11-24-2021, 03:44 PM   #12
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default

Quote:
Originally Posted by shosty View Post
I really know next to nothing about developing but I think it's good that they use that as a base to be honest. The flip side is that if they develop for Arch then it's quite possible that only people on bleeding edge will be able to use them and I think that number is less than those using more conservative distros. Most linux savvy devs I've seen (VCV Rack being a good example) use an old toolchain to build against because if it works on old versions of packages/libraries/compilers then most of the time it will work with new versions too. The reverse is much more of a gamble and for binaries it can result in them not running at all.

The problems with running more bleeding edge distros aren't confined to closed source stuff though, I don't know if it's still an issue but there used to be VST versions of the gxplugins which wouldn't show their UI in hosts when using Arch because of a fundamental library version difference (I don't remember which it was now).
You make a good point. It's not an easy answer. And, if really smart people like the developers struggle to find the right answer, I can't expect much from myself. But I do admit that I see why it is so problematic for Linux.

However, the thought occurs to me that if developers compile just twice for linux, they could get the Ubuntu crowd as well as the Arch crowd. But then, their goal is to not have to spend so much time compiling. In the thread I referenced above, a developer gave a example of what goes into compiling for a binary release. It was something like this (I'm paraphrasing, but you'll get the gist):

Windows x64 VST3
Windows x32 VST3
Mac ARM M1 VST3
Mac Intel VST3
Mac ARM M1 AU
Mac Intel AU
Linux i686
Linux AArch64
Linux ARM v7
Linux X64 (Ubuntu)

The above would feasibly be ten separate binaries if intel and the ARM architectures alone are supported. But it's not that simple. Each distro has different library versions depending on how conservative or bleeding edge the distro is. And, each distro has multiple potential architectures. Let's say that Developers choose to not support 32-bit architectures, and only support the Mac M1 ARM architecture, they would still have a lot of binaries to create:

Windows x64 VST3
Mac ARM M1 VST3
Mac Intel VST3
Mac ARM M1 AU
Mac Intel AU
Linux X64 VST3 (Ubuntu)


If the developers can't find a way to reduce the number of binaries they have to create, and if they support just the main distro families with binaries, it could feasibly balloon to this:

Windows x64 VST3
Mac ARM M1 VST3
Mac Intel VST3
Mac ARM M1 AU
Mac Intel AU
Linux X64 VST3 (Debian Family)
Linux X64 VST3 (Fedora Family)
Linux X64 VST3 (OpenSuse Family)
Linux X64 VST3 (Arch Family)
etc.
etc.

The problem is, that's still not sufficient to guarantee 100% compatibility, because distros within the same family can have different library versions. If one were looking at it clinically, one could justifiably argue that each separate distro should have its own binary. For example, a binary for Debian, a binary for Ubuntu, a binary for Mint, etc.

As you can see, binaries are an exponentially growing headache for developers. Let's say that it takes 30 minutes to compile a binary. Let's say that the developer has multiple computers, each running a VM for a clean binary. It could still possibly take days to babysit everything. It's just not feasible.

The Proprietary Audio Plugins for Linux thread discusses some of these problems. One of their current strategies is to use the least number of dependencies as possible. Most of these dependencies come just from the GUI. App compatibility is likely the chief reason Chris from Airwindows doesn't use GUIs. But Chris' plugins are with a small number of parameters and will work without a GUI. But some plugins use hundreds of parameters, and it's just not possible to efficiently use a VST with no GUI and 600+ parameters. It becomes a real problem.

It may not be so bad, if Linux only used a single desktop environment, but there are 20+ desktop environments and windows managers and such. Available libraries that could potentially cause dependency hell are numerous.

And so far, I've only mentioned VST3. I've left out LV2, AAX and other plugin formats.

So, As I understand it, the thread above tries to find best practices to achieve the most compatibility with a reasonable number of binary builds. Even then, there are no guarantees. It's very complicated and up to bigger brains than my own.

Last edited by audiojunkie; 11-24-2021 at 03:51 PM.
audiojunkie is online now   Reply With Quote
Old 11-24-2021, 04:56 PM   #13
flechtwerk
Human being with feelings
 
Join Date: Oct 2020
Posts: 32
Default

This is a very good blog post on the subject (though not about music software specifically). It's highly critical of solutions like Flatpak etc. where each package ships with its own libraries and opts for general backward compatibility of libraries instead:
https://ludocode.com/blog/flatpak-is-not-the-future
flechtwerk is online now   Reply With Quote
Old 11-24-2021, 06:20 PM   #14
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default

Quote:
Originally Posted by flechtwerk View Post
This is a very good blog post on the subject (though not about music software specifically). It's highly critical of solutions like Flatpak etc. where each package ships with its own libraries and opts for general backward compatibility of libraries instead:
https://ludocode.com/blog/flatpak-is-not-the-future
Cool! I'll happily read it! The more I understand this stuff, the better!
audiojunkie is online now   Reply With Quote
Old 11-25-2021, 05:34 AM   #15
flechtwerk
Human being with feelings
 
Join Date: Oct 2020
Posts: 32
Default

The solution laid out in that blog post would be that once a binary runs stable in a more conservative distro like Ubuntu, it should also run in a bleeding edge distro since the dependency libraries are backwards compatible. (Which used to be not the case in Linux but is being taken care of more and more - a notable exception being, according to the blog post, the GTK toolkit.)

Of course there remains the question of how to obtain the binaries, and I understand that companies like Bitwig might be worried about the "official" method of installing their commercial product in a distro like Arch is via the AUR - so packaging is taken care of by the user base, with all the security dangers this entails. But as one of the Arch users I would argue that this is not a problem - there is trust in the community and also trust in one's own judgement. The packaging process is transparent after all and one can quickly glance over the installation scripts to detect whether they are doing what they are supposed to do. (Many Arch user wouldn't trust commercial packaging systems anyway that would hide what they are doing.) In the end the installation and updating process is very convenient, I'd prefer it over the Windows way of constantly downloading and installing new versions of Bitwig or Reaper etc. individually any day.

One the question of distros from I user perspective I'd say that Arch (or possibly its derivates though I have no real experience with them) is by far the best distro for audio production if one is a bit tech savvy and willing to put in the time. And otherwise there is something like Ubuntu Studio which works quite perfectly out of the box, in my experience. If companies could develop for these more conservative distros and then the community of the rolling release distros would take care of reasonable backwards compatibility, things could maybe get quite smooth for the developers.

Last edited by flechtwerk; 11-25-2021 at 05:35 AM. Reason: typo
flechtwerk is online now   Reply With Quote
Old 11-25-2021, 06:27 AM   #16
wastee
Human being with feelings
 
Join Date: Mar 2015
Posts: 73
Default

Nice share!
wastee is offline   Reply With Quote
Old 11-25-2021, 03:26 PM   #17
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default

Quote:
Originally Posted by flechtwerk View Post
The solution laid out in that blog post would be that once a binary runs stable in a more conservative distro like Ubuntu, it should also run in a bleeding edge distro since the dependency libraries are backwards compatible. (Which used to be not the case in Linux but is being taken care of more and more - a notable exception being, according to the blog post, the GTK toolkit.)

Of course there remains the question of how to obtain the binaries, and I understand that companies like Bitwig might be worried about the "official" method of installing their commercial product in a distro like Arch is via the AUR - so packaging is taken care of by the user base, with all the security dangers this entails. But as one of the Arch users I would argue that this is not a problem - there is trust in the community and also trust in one's own judgement. The packaging process is transparent after all and one can quickly glance over the installation scripts to detect whether they are doing what they are supposed to do. (Many Arch user wouldn't trust commercial packaging systems anyway that would hide what they are doing.) In the end the installation and updating process is very convenient, I'd prefer it over the Windows way of constantly downloading and installing new versions of Bitwig or Reaper etc. individually any day.

One the question of distros from I user perspective I'd say that Arch (or possibly its derivates though I have no real experience with them) is by far the best distro for audio production if one is a bit tech savvy and willing to put in the time. And otherwise there is something like Ubuntu Studio which works quite perfectly out of the box, in my experience. If companies could develop for these more conservative distros and then the community of the rolling release distros would take care of reasonable backwards compatibility, things could maybe get quite smooth for the developers.
Very interesting insights! And the blog post was quite informative as well. I would go so far as to suggest that the reason developers are choosing Ubuntu as their main proprietary distribution distro is because it is conservative. I hadn’t thought about that prior to the blog post and your posts but it makes sense because arch is so bleeding edge.
audiojunkie is online now   Reply With Quote
Old 11-25-2021, 08:19 PM   #18
MiddleC
Human being with feelings
 
Join Date: Apr 2021
Posts: 41
Default

https://www.youtube.com/watch?v=Pzl1B7nB9Kc

Linus Torvalds on this topic...
__________________
When my mixes sound shallowly pessimistic, I apply the optimism gain stage and move on with my life.
MiddleC is offline   Reply With Quote
Old Yesterday, 09:16 AM   #19
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default

Quote:
Originally Posted by MiddleC View Post
https://www.youtube.com/watch?v=Pzl1B7nB9Kc

Linus Torvalds on this topic...
Well said! Linus really gets to the point. :-)
audiojunkie is online now   Reply With Quote
Old Yesterday, 10:07 AM   #20
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default

So, as far as dealing with Proprietary/commercial/closed-source binaries across distros on Linux, current thought argues for/against the following strategies:

A. Build a separate binary for every OS, chip architecture, or distro release. This is the Original problem and currently the only current guaranteed way to make sure everything will work. It is also so counter productive that NO ONE will do it.

B. Develop something like the OBS (Open Build System) https://build.opensuse.org/ But cover every OS, chip architecture, or distro release. The OBS seems to be really cool and has a lot of potential, but doesn't support everything that is needed, and has its own set of problems.

C. Listen to Linus when he says: Don't "Break Userspace!" In other words Library developers need to maintain 100% backwards compatibility No Matter What, and distro developers need to stick to this core set of compatible libraries.

D. Containerization such as Flatpak, SNAP, etc.

E. While waiting for better solutions to come along, use the recipe for hosts and plug-ins to play nicely together:

1. Keep dependencies minimal (for both host and plugins) and don't expose unnecessary symbols in either (-fvisibility=hidden etc, and don't assume that just 'stripping' a binary does anything of the kind)
2. Dynamically link to system libs in preference to requiring custom library versions or using LD_LIBRARY_PATH hacks
3. Build to the earliest version of system libs you want to be compatible with (most APIs will maintain compatibility in future versions, the reverse is seldom true)
4. Statically link, only if you absolutely have to, and if its compatible with licenses etc.
5. Hosts should load plug-ins with dlopen(RTLD_LOCAL | RTLD_LAZY) as they already do, and that should be enough to keep plug-ins namespaces segregated from one another. There are still some edge cases, like host application vs plugin symbol collisions, but most of these turn out to be rare if the other suggestions are adhered to.

"Then you have something which works at least as well as it does on other platforms. (Intentionally or otherwise, this seems to be what happens with for example Reaper on Linux, the host has minimal dependencies and seems to dynamically link to only a few system infrastructure libs, and by far the vast majority of Linux plug-ins appear to work without problems)."

In addition to this, the Linux binary needs to be compiled for a conservative distro using the older libraries, with the expectation that it should mostly work on a newer distro, like Arch, with bleeding edge binaries.

I would add to this that in all fairness, commercial developers need to provide a way for users to test the software on their own distros by providing a way to demo the software on their personal distro. Refund requests are probably the reason why developers are afraid to label the their software anything other than "experimental" or provide more than limited Linux support. But that doesn't encourage the customer to buy software, even if it limits their liability. There needs to be a way for the consumer to feel more safe that their purchase will work on their distro. Labeling the build as supporting only Ubuntu cuts into their sales.

Honestly, I don't know any of the answers. I do understand the problem much better now though, and I see that there is no easy solution as it stands. Hopefully a solution that far outweighs the likely included cons will be presented for proprietary binary distribution for Linux. Right now, there is not a perfect solution, and it benefits us as consumers to understand the point of view of the developers.
audiojunkie is online now   Reply With Quote
Old Yesterday, 01:20 PM   #21
4duhwinnn
Human being with feelings
 
Join Date: Mar 2017
Posts: 670
Default

Ages ago, Puppy linux developed .sfs files, which contain the app(s) and needed libs, utilities etc, and the .sfs files can be loaded on the fly. There was one that contained zynaddsubfx, hydrogen, jackd and qjackctl, among others, so users of a small nimble non-media-centric distribution, could load a working audio production environment whenever-wherever, as Puppy is very physically portable.

Long ago, Alexander Bique nearly single-handedly took the great U-he instruments and effects library, and made them each installable by a script, and Rui Capella sussed out the issue of multiple plugins being provided in one installer...

Per those examples, the issue is really one of human nature. The incumbent stubborness and selfishness of so-called 'free' software developers over-riding common sense, and what benevolence survives 1st world indoctrinations, all in the name of freedumb (aka I'll do it my way, so the 'lesser' distros and apps and libs can all go pound sand).

As for me, I use one distro for audio editing, another for sessions using strictly commercial audio tools, another for mixed-license sessions, another for printing/office duties, and another for travel. (I'll soon add a 6th, for Fedora Pipewire, and see how the dust settles)

I have and use a lot of delightful plugins, so juggling a mere 5 distros is trivial, and major issues popping up are almost nonexistant. For those who will stick to one distro on one computer, sojourns into dependency hell can be minimized by an hour or four in a package manger gui, making a carefully stripped-down distro sans the typical linux shovelware. Only keep what you actually use.
Cheers
4duhwinnn is offline   Reply With Quote
Old Yesterday, 01:24 PM   #22
4duhwinnn
Human being with feelings
 
Join Date: Mar 2017
Posts: 670
Default

As a counterpoint to the above, I'm pretty sure the Surge coding team
members never shared the same crib when growing up, and are modeling cooperation, efficiency, common sense, and communication, in the arena of genious, extremely well. An example distro creators/maintainers might do well to ponder.
4duhwinnn is offline   Reply With Quote
Old Yesterday, 02:49 PM   #23
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default

Quote:
Originally Posted by 4duhwinnn View Post
As a counterpoint to the above, I'm pretty sure the Surge coding team
members never shared the same crib when growing up, and are modeling cooperation, efficiency, common sense, and communication, in the arena of genious, extremely well. An example distro creators/maintainers might do well to ponder.
For about 15 years, we had the LSB (Linux Standard Base), but it died in 2015. It would sure be nice if everyone would play well with each other. Unfortunately, I don't think it's something that I can bank on. In the meantime, I think developing a set of strategies on how to best deal with the situation as it currently stands is our best bet. Using comparison tools like ldd is very useful. I wonder what else is out there.
audiojunkie is online now   Reply With Quote
Old Yesterday, 02:51 PM   #24
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default

I discovered this document that relates to the whole conversation at hand:

https://itvision.altervista.org/why....p.current.html

Major Linux Problems
on the Desktop, 2021 edition

It's regularly updated and contains a lot of what we've been discussing.
audiojunkie is online now   Reply With Quote
Old Yesterday, 05:40 PM   #25
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default

I've been going back and reading the comments to the blog post, "Flatpak Is Not The Future", and they have been interesting. There are some good arguments FOR containerization. Some of the comments:

Quoted Post:

I’m sorry but I’m going to have to disagree strongly with you, for the same reason that “sysvinit/OpenRC/etc. is all you need” anti-systemd people are wrong. RPM and DEB are only good enough if your use-cases are circumscribed by the needs they were originally developed for.

To some extent, the containerization side of Flatpak is EXACTLY like systemd, in that it’s a “we’ve given you twenty years to fix this. We’re tired of waiting” response to people who preach about “the right way to do it” and then fail to get sufficient traction out of initiatives like the LSB.

For example:

1. I want to run an LTS distro, but I also need one or two newer apps. My choices are a PPA, which is the wild west as far as trust goes, or Flatpak, which provides a centralized system of sandboxing, allows me to easily customize the sandboxing manifest, and, if it’s Flathub, approval isn’t *fully* automatic like for PPAs.

2. Ubuntu has decided to stop packaging Firefox with APT in favour of official Mozilla snaps. Mozilla also officially maintains their Flatpak and binary tarballs with automatic updates… which one has sandboxing?

3. I want to sandbox as many desktop applications as possible for improved security. Which is less likely to cause me strife: Retrofitting other people’s packages using Firejail or just tweaking the Flatpak installs using Flatseal?

Beyond that, I’m reminded of articles like Drew DeVault’s “Developers: Let distros do their job” which say “Ship your software as a simple tarball. Don’t ship pre-built binaries” and “One thing you shouldn’t do is go around asking distros to add your program to their repos. Once you ship your tarballs, your job is done. It’s the users who will go to their distro and ask for a new package.” and brush under the rug the effect it has: “We developers should be elite gatekeepers over everyone who doesn’t know how to compile software from source”… imagine if audiophiles got to have veto power over every product targeted at the market segment of people who don’t know how to hook up a multi-component hi-fi system!

"This is apparently working as intended. They want runtimes to be a free-for-all, filling your hard drive with gigabytes of custom junk for every app."

No, this is “Red Hat/Debian/whoever didn’t make an RPM/Deb/etc. for the app I wanted” all over again… literally. Flatpak is just less willing to foist “Run alien and pick up the pieces” off on the end users in the 21st century.

"Today, on my machine, the KCalc Snap takes a full seven seconds to start up. Not just the first time after boot; every time, without fail. Seven seconds to start a calculator."

No argument. Snap’s architecture is garbage, even by containerized application standards.

"Flatpak allows apps to declare that they need full access to your filesystem or your home folder, yet graphical software stores still claim such apps are sandboxed."

Do I really need to start enumerating things GNOME gets wrong in other spheres?

The Flatpak CLI is the first-party UI and it gets announcing required permissions right. The KDE Discover plugin doesn’t make a half effort like that and just presents Flatpak packages with the same UI as native packages. Don’t be that infamous anti-Flatpak FUD site I won’t give free publicity to.

In short, this is another case of “WORKSFORME/WONTFIX: Your needs are invalid”, just like the last two decades that inspired Flatpak and snaps and AppImage and the like.

"This is the most complicated and brittle way to implement this"

The passage you quoted says “meaning that applications don’t need to do any additional work”. “Don’t need”, not “shouldn’t”. Emphasis mine.

It also omits the next sentence: “Applications that aren’t using a toolkit with support for portals can refer to the xdg-desktop-portal API documentation for information on how to use them.”

That sounds like a pretty standard “Don’t reinvent platform APIs if you can work within them” that would also be appropriate for something like ensuring reliability on the Linux Standard Base or SDL or any other means of ensuring forward compatibility with the proposed paradigm.

This argument sounds like a disingenuous case of “Any solution but mine must run up against a heavy incumbent advantage that favours my solution”. Retrofitting the platform APIs is a very elegant way to get most of the way there with little to no demand on each individual upstream developer’s time and the main reason it works as poorly as it does in my experience is that GtkFileChooserNative is a young API.

"How can Fedora justify publishing apps while masquerading as the upstream developers?"

How is this any different from the mess with Debian vs. Fedora RPM vs. OpenSuSE RPM vs. etc. when it comes to package naming? A good tool can’t magically fix a bad maintainer.

Don’t play the nirvana fallacy card when APT/RPM, Linux itself, and UNIX before it, are perfect examples of “good enough” solutions running rings around attempts to attain perfection.

The post even mentions “Worse is Better” in the next section, so I get a strong sense that this is a disingenuous article, not arguing in good faith. As I said before, people who think like you have had 20 years to come up with a solution that meets the needs Flatpak has set its sights on.

"Proponents of containerization believe it is the solution to every problem, the hammer for every nail."

More an acknowledgement that Linux can’t afford to ignore the “Windows won because it was backwards compatible with DOS. Later Windows won because all 32-bit versions of Windows are still compatible with 16-bit Windows apps. etc. etc. etc.” network effects.

Flatpak is an acknowledgement that you only argue for a clean-slate approach to sandboxing if you *want* it to fail.

Not pictured: Steam and the game running directly on the user’s native environment, like they do on every other OS.

"Not pictured: Steam spyware-ing the user’s DNS cache and doing other shady things in the name of “anti-cheat”."

"A major goal of most of these technologies is to support an “app store” experience […] Flathub only says they don’t process payments at present. It’s coming."

This is willfully blind at best.

Canonical already tried this using traditional APT-and-Deb packaging, no snaps needed. Containerized packaging is orthogonal to attempts to monetize.

All you need is a repository-based package manager (APT) and suitable metadata (AppStream)… both of which are requirements completely independent of these new packaging formats.

"Just in the past week, I’ve used several Linux apps distributed as plain binary tarballs that run on my native environment."

Oh, like my Humble Bundle games where, to this day, I have to root around through old Ubuntu/Debian package archives for libraries with the right `.so` version or google up answers for why they crash mysteriously on startup?

"GOG.com’s strategy for Linux support is just about the opposite of Steam’s, and works just like traditional installers for Windows."

…and every time I upgrade my Ubuntu, I’ll have at least one or two games where I have to go in and delete one of the bundled libraries to resolve a “crashes on startup” issue.

Not to mention, vintage games don’t have the “security updates are being released that need to make it to the end users promptly” issue.

As a retro-computing enthusiast, this reminds me of the Good Old Days™ of Windows 9x when malware was in its infancy and gamers hadn’t yet started to trickle away to the convenience of consoles before Steam made things more convenient.

"Imagine if millions of office workers used Excel on Ubuntu. How loudly do you think businesses would complain if a distribution upgrade broke it?"

An “Easier said than done.” argument. Where do you draw the line on which libraries are safe to assume will be present in perpetuity? How do you get distros to agree on it? How do you ensure they keep offering access to the same ancient versions forever? What if upstream don’t properly follow SONAME versioning for ABI compatibility? (Automatic analysis has shown humans suck at that)

How do propose that we *now* get all the distros agreeing on dependency metadata for packages they consider beneath their notice when they weren’t before.

Again, the phrase that defines Flatpak’s raison d’etre… “things the open-source ecosystem is failing to deliver on”… just like why systemd replaced sysvinit.

"The GTK problem"

I have no argument with this. As a KDE user, I agree that the “and pre-installed on the vast majority of Linux desktop” makes his argument accurate.

…I won’t use GTK 3.x now that it’s grown too many GNOME-isms and still lacks stuff Qt provides by default like minimal toolbar customization, but I agree with the idea to arm-twist GNOME into being better stewards of it.

"Is Flatpak Fixable?"

Let me know when you manage to get sufficient buy-in on that “declare library dependencies that don’t care about Debian vs. Fedora vs. OpenSuSE vs. Arch vs. … naming conventions”, “Convince all app developers to retrofit their apps for a new API”, and “Deprecate install-time permissions so aggressively as to kill off any momentum already gained”.

Final Verdict: Talk is cheap. Prove you can do differently than the last 20 years of APT and RPM that inspired these packaging systems and I’ll listen.
audiojunkie is online now   Reply With Quote
Old Yesterday, 06:08 PM   #26
audiojunkie
Human being with feelings
 
audiojunkie's Avatar
 
Join Date: Nov 2011
Posts: 286
Default

I suspect that while Flatpak may not be the complete solution, and maybe not the solution at all for plugins, it will be a solution for some commercial software. (I personally don't like SNAP or Canonical's planned store monopoly, so I'm not even considering it.) Which brings us back around to my opinion that the developers have a big struggle on their hands and a problem with no easy answer. I can understand why proprietary (commercial) binaries haven't taken off on Linux. I still hope to see things get better, and I can see for myself that the developers are currently doing the best they can, given the circumstances, to make grow Pro Audio on Linux. I have nothing but absolute respect for Linux audio software developers!!

Which brings me to another thought. In the Linux realm, as it stands now with the problems developers have with supporting commercial software across large numbers of distros, it makes me consider WINE again. WINE, aside from the occasional regression, is continuously getting better, and continuously supporting more and more commercial applications. I don't expect this trend to change. I've (so far) avoided everything with WINE and focused on native applications. I've always felt that this is the most stable way, and that supporting native apps will continue to improve the situation as developers see that Linux consumers want commercial products on Linux. However, in light of what I am learning, I am also suspecting that while the number of proprietary applications will continue to grow (and the growth, I suspect will become progressively faster as well), I think it won't grow as fast as I'm wanting, and maybe I need to consider, for the time being, being willing to use WINE.

After all, to quote WINE's main page, WINE is:

"a compatibility layer capable of running Windows applications on several POSIX-compliant operating systems, such as Linux, macOS, & BSD. Instead of simulating internal Windows logic like a virtual machine or emulator, Wine translates Windows API calls into POSIX calls on-the-fly, eliminating the performance and memory penalties of other methods and allowing you to cleanly integrate Windows applications into your desktop."

There is almost no loss in speed, and compatibility improves daily.

I'm now of the opinion that we should freely use whatever is available for us, whether that be native apps from developers who code to best practice strategies for binary compatibility, containerization such as Flatpak, or compatibility layers such as WINE. As of now, there is not really a truly best solution.

Another thing that I have realized is that where the musician plans to obtain the bulk of his music-making tools (commercial vs open source) matters a great deal in what distro I would recommend to a new user. Right now, I am firmly coming to the belief that there are really only two distros worthy of consideration Ubuntu and Arch (and by way of Arch, I am including OSes binary compatible with Arch, such as EndeavourOS or Manjaro. If a user just wants to make music and doesn't want to deal with library compatibility problems, these are the best two choices. Ubuntu, for those who want to purchase proprietary plugins and software, and Arch for those who want the highest number of open source packages available on a distro for music creation. Granted, advanced users can pretty much use any Linux distro and pull the binaries and manually make the libraries work, but most music producers just want to make music and not worry about the details of the distro.

Over all, so very interesting articles and developer comments on the state of proprietary software in the Linux world!
audiojunkie is online now   Reply With Quote
Old Today, 12:12 AM   #27
4duhwinnn
Human being with feelings
 
Join Date: Mar 2017
Posts: 670
Default

Another way to look at such problems, is that there are actually so many solutions from disparate sources, that are good enough in actual use, that a massive concerted unified centralized effort to close the gap on dominating 'the desktop' is only going to make things marginally better, because the technology margin in the arena of finishing tasks, is not all that big.

One devil in the details is that kids all use mac and win systems at school, and apart from android icon plucking on phones, and a few tablets, see little need to learn a new (linux) system. They are immersed, with no need to dry off.

A second devil, is the myth of software without a $price tag, somehow being 'free'. A developer with a day job, who is also coding a side-job linux project is charging (time-is-money) their family and friends and community, the high price of lost family time, and beneficial personal involvements, and quite possibly their own long term physical and mental health will suffer. The passion for coding, like any other after-work activity, needs to be carefully managed. Even un-married singles fresh out of uni, need healthy friendships, and those take time to develope and maintain.

Conversely, can you imagine if the top 500 linux coders had been paid market rate wages by a pair of corporate giants, for 50 hours per week for the last 5 years, how great the linux systems they would produce might be?

(Or maybe we would just have Systems E, F, G, H, I, and J...following in the
footsteps of System D?)
4duhwinnn is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 11:31 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.