When building plugins for Linux there are two options and one problem:
- Link to the system libraries
- Build all your dependencies and statically link to them
- Can't use gtk or qt.
- It forces to build using an old system in order to maintain a maximum level of compatibility. Which prevents from using modern compiler and library features.
- There is no guarentee that the plugin will load on the user system.
- It requires work to maintain your development libraries, some might not even be working on the user system (paths with fontconfig?, configuration files format change, ...).
- More work to ensure maximum compatiblity toward libm, libc, ...
- The host will probably use dlopen(RTLD_LOCAL), because RTLD_DEEPBIND creates various issues, breaks most plugins and is not used in the end. RTLD_LOCAL means that the plugin will use the symbols from the host before its own, which might turn out to be bad for multiple reasons. While if everyone uses the symbols from the system's runtime there is no conflict at all. One problematic scenario is:
- host is staticaly linked against libX version N
- plugin is dynamicaly linked against libX version N+1
- libX_blah_create() returns an opaque pointer libX_ptr, which has a different data layout in version N+1
- libX_doY(libX_ptr) is a new symbol introduced in version N+1 which takes this opaque pointer as argument; unfortunately in our case it was allocated by libX_blah_create() from version N, but then used by the symbol libX_doY() from version N+1 which expects a different data layout -> BOOM
Yes absolutely! And, this is the biggest problem. This is so big that it is unrealistic. Right now a plugin needs to go through a huge pain in order to show a GUI. The reason why it can't use gtk or qt, is because those don't behave well when mixed together. And imagine multiple version being mixed together having symbols clash...
The consequence is that the plugin developper will use xcb, cairo, and basically write their own toolkit in order to display a knob. No one wants to write a GUI toolkit in order to write a plugin.
- Be able to run the plugins in their own process.
- Plugin may not use an ABI but a protocol involving IPC.
- There should be a registry of all the plugins available, to which you can communicate and create instances of a plugin.
I will focus on the idea that DAWs an plugins will be distributed via flatpak and linked to its runtime. Obviously it does not make flatpak a requirement but instead forces the design to work across sandboxes and runtimes.
Plugins and DAWs may be linked statically or against a flatpak runtime, it does not matter as they will be isolated from each others.
Then, we need the plugin registry.
- it must be reachable from any sandboxes
- it must be able to index plugins available on the system and installed via flatpak
- it must be able to spawn plugin processes and establish IPC between the DAW and plugins
A proprietary plugin can:
- Link dynamically against (system|flatpak)'s runtime or link statically
- Use GTK or Qt (or whatever they like)
- Be distributed via flathub
- Be used by a DAW built against a different runtime
The plugin and daw can still implement vst2, vst3, lv2, ..., and it would be implicitely wrapped by a transparent host doing the daw <-> vst3 <-> bridge/plugin <--- IPC ---> bridge/host <-> vst3 <-> plugin.
We don't want to impose to the industry to support a new plugin api.
This will let a DAW sandboxed and started using runtime Y to use plugins from the system or even built for an other flatpak runtime.
In the end this work will turn Linux into a top level place for multimedia processing:
- Painless GUI using gtk/qt or whatever.
- Easy to develop
- Easy to debug
- Easy to distribute and get visibility in Flathub
- Easy to get a working system and stable system for the user
- Preserve user's freedom and control by linking dynamically to the flatpak runtime: the user has control over his runtime
- Preserve user's security by having different sanboxing parameters for each plugins and daw. Otherwise the DAW must have the maximum permissions requirements from each plugins, and it each plugins would inherit the daw's permissions which is not what we want, for example we may want to disable network access for most audio plugins.
The overhead comes from context switching between processes, and the smaller the latency is (undestand audio interface buffer size) the higher the overhead becomes.
The solution is to bulk process.
Let's take a concrete exemple:
- DAW is using runtime X
- N plugins from the same vendor We could create a single plugin process for multiple plugins from the same vendor. That way, when the host host can group the processing requests toward nultiple plugins into one single bulk request, reducing the amount of context switch.
Perhaps unorthodox idea, but why not build tailored Wine to the DAW? Like Steam does for games.
I personally use Windows versions even of the plugins for which there is a Linux version, just because it management wise easiest way to get shit done (when producing music). Other options is to have ~/.wine, ~/.vst, ~/.vst2 and what not.
All platforms considered it would be cool if plugin industry would give a shot doing something similar Sun tried to do years back with PWI (Public Windows Interface). It was a failure but in audio plugin it could work out. I.e. standardize a subset of Windows API that plugins would use. That could even mean ABI compatibility for plugin binaries using the same ISA
I.e. instead of macOS, Windows and Linux version of a plugin you'd just have ARM, x86 etc. versions
Plugins are in the just huge calculators with an UI, pretty simple stuff, when you think it that way. I/O wise they are most of the part as simple as Minesweeper game.
Plugin API's do not need any change. What is needed is a standard PRE (Plugin Runtime Environment) :-)