Old 01-26-2020, 12:16 PM   #1
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default Prototype for Preset Management

Old concept has been abandoned. This is the new one, more info at the end


================================================== ===========================
Hi all,
its my first post for Reaper, just started a few month ago to work with it.

Wanted to ask for feedback, if my idea is worth continuing.

I build a rough prototype for browsing Halion presets that shows all preset on a 2D graph. The presets are ordered by similarity via a machine learning algorithm (tsne) and I am working to improve this.
You can prelisten every sound (each dot is a preset) and directly load it into reaper. The video is linked via dropbox( hope tats OK to everyone).
Idea behind it is, that I am trying to build a more musically oriented browsing and to find out how good I can map the sounds by similarity with an algorithm. So if you find a sound (or know it) you can search its surrounding for similar sounds.

The idea is not new, currently XLNaudio does this for drum sounds with XO and this seems to be based on a google paper https://experiments.withgoogle.com/drum-machine

But until now, no one has done this for synths and there is no product out, so I thought to give it a try.

Current status:
- LUA Script to render a folder of presets into a folder of wav files
- Python scripts to extract audio features from wavs(very basic)
- Pyton script that draws the plot, plays the sounds and loads presets into Reaper

Here is the video, hope it works
https://www.dropbox.com/s/mw6emdexn6..._Demo.mp4?dl=0

Would be very interested if you find this useful.

Thanks a lot!
Attached Images
File Type: jpg Reaper_tsne.jpg (45.5 KB, 790 views)

Last edited by dimentorium; 04-22-2020 at 12:59 PM. Reason: Complete new concept
dimentorium is offline   Reply With Quote
Old 01-26-2020, 12:22 PM   #2
schwa
Administrator
 
schwa's Avatar
 
Join Date: Mar 2007
Location: NY
Posts: 15,749
Default

Nice work, it must have been fun to work on!
schwa is offline   Reply With Quote
Old 01-26-2020, 01:25 PM   #3
Jae.Thomas
Human being with feelings
 
Join Date: Jun 2006
Posts: 22,567
Default

this looks really neat!!!
Jae.Thomas is offline   Reply With Quote
Old 01-26-2020, 01:26 PM   #4
Jae.Thomas
Human being with feelings
 
Join Date: Jun 2006
Posts: 22,567
Default

so how would one populate all of this? There's a script that runs through presets? Obviously it would need to have one type of scrollable preset system right?
Jae.Thomas is offline   Reply With Quote
Old 01-26-2020, 01:42 PM   #5
Xenakios
Human being with feelings
 
Xenakios's Avatar
 
Join Date: Feb 2007
Location: Oulu, Finland
Posts: 8,062
Default

Quote:
Originally Posted by Jason Brian Merrill View Post
so how would one populate all of this? There's a script that runs through presets? Obviously it would need to have one type of scrollable preset system right?
Yeah, the problem I see with this is that there's no universal way in plugins to programmatically change presets. (And there's no way to be sure when the plugin has actually fully changed the preset so that it is ready to offline render audio...) Still, a very cool thing.
__________________
I am no longer part of the REAPER community. Please don't contact me with any REAPER-related issues.
Xenakios is offline   Reply With Quote
Old 01-27-2020, 01:28 AM   #6
brummbear
Human being with feelings
 
brummbear's Avatar
 
Join Date: May 2016
Location: out west
Posts: 301
Default

Looks quite useful to me. Besides XLN Audio Arturia also uses machine learning to find "similar sounds" in Analog Lab4.

While there is indeed no universal way of scanning through all presets across plugins I think your browser would work really well over NKS previews! These typically contain thousands (if not 10's of thousands) preset sounds only for Native Instruments Komplete Kontrol series. There are quite a few other plugins available by now from other vendors that also come with NKS support. And then there are also dozens of plugins which have received NKS support by users in the meantime. Finally, there is an open source tool "PresetMagician" (https://presetmagician.com/) which automatically extracts presets incl audio previews from many paid and free plugins. I have used it to generate more than ten thousand audio previews of non NI plugins. Your browser would be a nice way of sifting through these vast sound libraries...
brummbear is offline   Reply With Quote
Old 01-27-2020, 07:20 AM   #7
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

Hi guys,
thanks for the feedback to answer the questions:

@Jason: for my test I created VST presets for Halion, so I simply go through the folder of presets and generate WAVS with the same name in a different folder. So if you have a bunch of preset files that reaper can load into the VST, this will work

@Xenakios: You are right! After trying some intelligent ways to get to presets with Halion (Halion only loads Multi Presets, which are not available), I ended up with a Python Mouseautomation script, that loads all programs and clicks the appropriate buttons. Not intelligent, but works for the test and was the fastest solution.
Next issue in Reaper with the loading, I simply waited some seconds. So rendering 5000 patches took around 20k seconds. But I did not need to be present

@brummbear: if the NKS files format is oepn readable, then this is definately possible.

So will continue to work on it and keep you posted ;-)
dimentorium is offline   Reply With Quote
Old 01-27-2020, 11:04 AM   #8
brummbear
Human being with feelings
 
brummbear's Avatar
 
Join Date: May 2016
Location: out west
Posts: 301
Default

Yes, just like wav all NKS audio previews (= rendered snippets of the patches) are simple readable ogg vorbis encoded files.

For automated preset/patch rendering I recommend you take a look at PresetMagician (linked above). At least for the already supported plugins (and there are quite a few!) you get all the audio rendering freely delivered. You can ignore all the NKS stuff - just grab the extracted audio files.

For any other plugin I believe there is no other way but via mouse/keyboard automation as you already did. In fact this is how some folks actually generate the sound previews for NKS libraries that are sold on the web (e.g. here: https://freelancesoundlabs.com/index...trol-vst-packs)
brummbear is offline   Reply With Quote
Old 01-27-2020, 01:02 PM   #9
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

Ok had a look into the demo from freelancelabs and as ogg files are supported by the python library I use, that would work as well.

Also did some rework on the classification (a good friend who is working in machine learning gave me some hints). The new version seems to cluster the sounds way better than before.

Hope it works again with the video
https://www.dropbox.com/s/xxk80mnn2x...NE_V2.mp4?dl=0

In theory this classification would also work on presets for effects if you render the sounds with the same input material (for example guitars or so).

Currently thinking what would be the best way to build a real prototype for distribution.
dimentorium is offline   Reply With Quote
Old 01-27-2020, 03:34 PM   #10
brummbear
Human being with feelings
 
brummbear's Avatar
 
Join Date: May 2016
Location: out west
Posts: 301
Default

Cool. The examples in your latest video seem to work really well.

Generally speaking, I believe that this type of clustering has a great future. Reason is simple: People are sitting on libraries of 10s of thousands of patches/presets spread across many different software instruments. Manufacturers have introduced more sophisticated ways of browsing over time, with NI NKS probably being the most advanced at this point especially when combined with their plugin control via Komplete Kontrol which also covers the parameter mapping alongside the browsing experience.

But all preset browsers irrespective of the vendor suffer from the effort required to tag the presets with actual language: Every preset has to be tagged manually by a human by all sorts of characteristics (say "trumpet, "airy", "polyphonic", "sampled", .... you name it). Without proper tagging presets cannot be found easily - even with prehear functions as no one will listen to thousands of sounds before making a decision.

Clustering along sonic similarity done by machines is an efficient substitute for the tedious manual tagging process employed nowadays.


For a simple prototype I would start with a stand alone application that only does the browsing:

* allow to add folder paths (or "directories") for scanning of rendered audio files. User is responsible for having meaningful file names of rendered audio files allowing to trace back to preset of an individual plugin
* UI: allow to select which folders will be considered in the (2D) plot
* UI: Play the sound and show the path/filename when hovering/clicking with mouse over point
(* UI: maybe allow some algo tweaking to modify clustering criteria? Could also mean we would need more than just 2 dimensional plots... e.g. also move forward backward in a 3D representation.)


In later stages for integration in existing browsing hosts you would return the path/filename to a host application which would use it to actually select a preset.
brummbear is offline   Reply With Quote
Old 01-29-2020, 01:50 PM   #11
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

Hi brummbear,
thanks for the feedback, had also a talk with a neighbor who is working as a composer. He told me kinda the same, that it takes him a lot of time to tag the synthesizer sounds and especially with large libs (Omnisphere) he does not really listen to a large bunch.

So I am currently testing with nksf and making it a bit nicer. So hopefully by end of the week I should have a script that can handle Halion and nksf in parallel ;-)

Will keep you posted.
dimentorium is offline   Reply With Quote
Old 02-09-2020, 05:17 AM   #12
todd_r
Human being with feelings
 
todd_r's Avatar
 
Join Date: Nov 2006
Posts: 855
Default

This looks amazing, would be fantastic if you could figure out how to make it universal
todd_r is offline   Reply With Quote
Old 02-09-2020, 01:55 PM   #13
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

So I am working to get the Python Scripts packed as an EXE to make it bit easier. Current status is.
- The script searches a folder for VSTPRESET, NKSF, WAV and OGG files
- If the names match, a preview file is mapped to the preset file
- The features from the audio files are extracted an store to harddrive
- The clustering algorithm is performed on the data

On the graph you see the presets and can listen to the previes. If the scripts finds a database in the folder, it will load the data and not scan.
Preparing the database takes around 2-3 seconds per preset.
So still a lot to go, but when I have the exe running you can test it if you want.
Next steps are then to add the tagging from the preset files and apply these to not tagged files
dimentorium is offline   Reply With Quote
Old 02-10-2020, 08:46 AM   #14
Ozman
Human being with feelings
 
Join Date: Feb 2015
Posts: 753
Default

I'd like to see this work with any chosen VST as well as one with sample folders (like the original script).
Reminds me somewhat of XLN - XO.
Ozman is offline   Reply With Quote
Old 02-10-2020, 03:02 PM   #15
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

After trying a lot, I can at least build the exe and run it on my system. Maybe someone else can try it (130MB).
https://www.dropbox.com/s/qloev3qszg1o38z/gui.exe?dl=0

Usage is simple. Just start when the GUI has loaded select a folder containing presets (vstpreset and nkfs) and audio (wav and ogg), then wait for quite a long time (~3 seconds per audio) and some more for the calculation. Then you get the 2d graph and by clicking on a dot the audio is played back. Would be really interesting if it works on other machines. My recommendation would be not too many files (it works with my 5000 Halion presets, but is slow).

If you want to render some presets here is ma reaper script
https://www.dropbox.com/s/bva4ep5257...Multi.zip?dl=0
Needs some folder change and a project with one track containing the vst and one midi note.

And before you ask why I use recursion... the wait function does not seem to be working inside a for loop, but with recursion I can stop reaper before rendering the project.

Will continue to work on this and see where its going.
dimentorium is offline   Reply With Quote
Old 02-13-2020, 01:39 AM   #16
brummbear
Human being with feelings
 
brummbear's Avatar
 
Join Date: May 2016
Location: out west
Posts: 301
Default

Quote:
Originally Posted by dimentorium View Post
Would be really interesting if it works on other machines...
Will continue to work on this and see where its going.
Thanks for sharing! Looks good (I tested in a Windows 10 sandbox, worked)! Really promising this :-)

The clustering was quite good for the sounds I tested with (1100 nksf/ogg presets for the Helm synth). It would be great if you made different classification criteria/parameters available to choose from e.g. spectral contrast - I assume you are using the librosa library? And maybe adding a third dimension for that matter like the Google folks did by adding color to the 2D plot...

By making different classification criteria available the community might be able to identify particularly useful combinations for clustering certain sound material.

I would also prefer to run it directly as a Python script rather than the big exe file...
brummbear is offline   Reply With Quote
Old 02-13-2020, 02:57 AM   #17
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

Cool, glad it works. Thoughs th EXE would be easier for most users not familiar with Python.

I am working on cleaning up the code, then we can have a look at the scripts.

Yes, I am using librosa for the feature extraction. Next step I want is to integrate the tag information from the preset files and use that information for coloring and also applying existing tags to not tagged sounds (but that implies the use of a neural network).

But I think this is a really interesting approach for using machine learning in music, so that the user can focus on making music and not searching thousands of presets.
dimentorium is offline   Reply With Quote
Old 02-16-2020, 01:35 PM   #18
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

So heres the link to my current repo.
https://dev.azure.com/ainysynth/_git/ainysynth

It should be accesible. Still a lot of refactoring and cleaning up/documentation before going to some new work here.
There is a requirements file with most of the dependencies. I am currently using the Python 3.7.6 for programming.

Would be good to know, if its accesible.

Last edited by dimentorium; 02-16-2020 at 01:46 PM.
dimentorium is offline   Reply With Quote
Old 02-16-2020, 02:41 PM   #19
nofish
Human being with feelings
 
nofish's Avatar
 
Join Date: Oct 2007
Location: home is where the heart is
Posts: 12,096
Default

Quote:
Originally Posted by dimentorium View Post
Would be good to know, if its accesible.
Yep, it's browsable online and cloneable (I just tried).
nofish is offline   Reply With Quote
Old 02-17-2020, 01:27 AM   #20
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

Cool! Thanks for testing.

I am open for any questions and feedback. Still a lot on my list I would like to cleanup to have a solid base for future ideas ;-)
dimentorium is offline   Reply With Quote
Old 02-18-2020, 01:25 PM   #21
brummbear
Human being with feelings
 
brummbear's Avatar
 
Join Date: May 2016
Location: out west
Posts: 301
Default

Hey Philipp,
Firstly let me make clear that I am not an expert on this topic... maybe my thoughts are not overly helpful.

Quote:
Next step I want is to integrate the tag information from the preset files and use that information for coloring and also applying existing tags to not tagged sounds (but that implies the use of a neural network).
I think this is a very interesting (and ambitious) approach. However, I would first try to hone the feature extraction & dimensional reduction, followed by optional clustering (with or without machine learning). One problem I anticipate is the quantity of the training data even with large tagged preset libraries. This may be compensated by high quality of the preceding feature extraction and (optional) clustering. If you decide to add clustering on top of dimensional reduction one could also have a classical algorithm to extract "dominant" tags for that cluster from pre-tagged libraries.

Either way, in my opinion the most interesting part for practical purposes would be browsing by sonic similarity (= the "humble" approach). Having neural network* created tags on top of that would be "icing on the cake". In order to get sonic similarity right there should be flexibility in the current project phase to play with the features that are extracted and used for dimensional reduction and clustering. As speculated earlier it may depend on the sound material what works best, e.g. percussive sounds would potentially benefit if you included spectral flatness to the extracted features.

As you mentioned in the beginning XO seems to make use of TSNE for its percussive sounds and you wanted to expand the concept also for synths. For a universal approach, maybe it would be useful to include a few more features from librosa in the initial extraction, then exclude non helpful features (which will depend on the sound material) before starting TSNE. Finally, an optional clustering process.

Manuel Sanchez uses GMM and SVM for clustering features and matching to tags. He uses it on biological data sets but this could work really well with audio too. Check this out: https://github.com/mscastillo/Classification

* a GMM or a SVM might be more suitable than a neural network for this task. In particular, a GMM could associate multiple characteristics, e.g. a sound can carry multiple tags like "trumpet", "airy", "pad".

Last edited by brummbear; 02-20-2020 at 05:56 PM.
brummbear is offline   Reply With Quote
Old 02-26-2020, 01:08 AM   #22
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

Hi Brummbear,
still on refatoring and will work on making the analysis part a bit simpler to extend with custom code. Did you manage to run the scripts on your computer?

I read through several papers and nearly all use the standard features (chroma, mfcc, ...) for identifiying sound thus I did not put too much brain into this part. Would also beliave that having the ADSR curve should be interesting to work on sounds.

And thanks for the link!

So hope I can continue by end of the week, this week was a bit busy.

By the way it would be interesting if users would send in their presets, to build up a large scale open database with synthesizer sounds for creating models and having them available to everyone.

Last edited by dimentorium; 02-26-2020 at 03:43 AM.
dimentorium is offline   Reply With Quote
Old 02-26-2020, 04:08 AM   #23
TonE
Human being with feelings
 
Join Date: Feb 2009
Location: Reaper HAS send control via midi !!!
Posts: 4,031
Default

Thanks for sharing.
Quote:
Originally Posted by dimentorium View Post
But until now, no one has done this for synths ...
Wrong , I did long time ago, it worked, however it is not everyone's style of use case, it requires combination of multiple external tools, I took Hierarchical Clustering Explorer*, then converting fxb preset banks to text form, with another tool, then making actions inside HCE triggering out the correct program change events, to reaper, so any HCE selection resulted in immediate correct sound selection in Reaper or elseDAW, you could play around, was good. As a learning tool it was interesting, if you want to compare multiple presets for their similarity or differences in their parameter combinations. Nowadays, taking even the audio output as image, something like spectral peaks views as exported .jpg/.png, if we would have it, you could even compare those.

* http://www.cs.umd.edu/hcil/hce/
TonE is offline   Reply With Quote
Old 02-26-2020, 04:18 AM   #24
TonE
Human being with feelings
 
Join Date: Feb 2009
Location: Reaper HAS send control via midi !!!
Posts: 4,031
Default

Quote:
Originally Posted by Xenakios View Post
Yeah, the problem I see with this is that there's no universal way in plugins to programmatically change presets. (And there's no way to be sure when the plugin has actually fully changed the preset so that it is ready to offline render audio...) Still, a very cool thing.
Bank select, program change? Not universal enough?

Command line tools allow anything, you could allow 1 second wait then after each preset selection, for example. Probably you are talking of huuuge sample libraries? For normal vsti synths it should work immediately without any waiting.
TonE is offline   Reply With Quote
Old 02-26-2020, 04:59 AM   #25
TonE
Human being with feelings
 
Join Date: Feb 2009
Location: Reaper HAS send control via midi !!!
Posts: 4,031
Default

Quote:
Originally Posted by brummbear View Post
Yes, just like wav all NKS audio previews (= rendered snippets of the patches) are simple readable ogg vorbis encoded files.

For automated preset/patch rendering I recommend you take a look at PresetMagician (linked above). At least for the already supported plugins (and there are quite a few!) you get all the audio rendering freely delivered. You can ignore all the NKS stuff - just grab the extracted audio files.

For any other plugin I believe there is no other way but via mouse/keyboard automation as you already did. In fact this is how some folks actually generate the sound previews for NKS libraries that are sold on the web (e.g. here: https://freelancesoundlabs.com/index...trol-vst-packs)
Why preset magician does not work with any vsti?
TonE is offline   Reply With Quote
Old 02-26-2020, 05:18 AM   #26
TonE
Human being with feelings
 
Join Date: Feb 2009
Location: Reaper HAS send control via midi !!!
Posts: 4,031
Default

Quote:
Originally Posted by dimentorium View Post
I read through several papers and nearly all use the standard features (chroma, mfcc, ...) for identifiying sound thus I did not put too much brain into this part. Would also beliave that having the ADSR curve should be interesting to work on sounds.

By the way it would be interesting if users would send in their presets, to build up a large scale open database with synthesizer sounds for creating models and having them available to everyone.
There is also sonic-annotator, command line brother of sonic visualiser.

Which is best tool for generating images from .wav/.ogg files? Which contain as much of their relevant features in that image, visually, meaning similar images (colors and shapes) should sound similarly. Spectral peaks view could be good candidate, but we can not generate .jpg/.png from Reaper.

The trick is finding best command line tools first, if already available somehwere. Then adding the missing rest.

For the audio to image transformation I could imagine a circular approach with a fixed length per circle, maybe 15 seconds? Or shorter? I guess 15s could work nicely, you could compare even 1 minute audio files, which would then contain 4 rounds in the circle, starting from the center going outwards the longer samples get, so images would also have different sizes. The longer the audio the bigger the radius, obviously, but the central comparison area would still be same/overlapping, so different size files can still be compared! And circular shapes look often beautiful because symmetric.
TonE is offline   Reply With Quote
Old 02-27-2020, 01:10 AM   #27
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

Hi TonE,
interesting tool the HCE, did not know that one.I would guess its using comparable algorithms that I am using.
So overall I would kindly ask to not start a discussion which tool to use or is better, but te keep the discussion which use case is interesting or useful. That will help me to focus the few time i have to develop useful stuff.

I tried several tools and code repositories and ended in the setup I have which is for my workflow the best I have. Also I tried to read papers and publications from people where I believe they are way smarter than I and use what is available to not redevelop the wheel.
dimentorium is offline   Reply With Quote
Old 03-01-2020, 09:06 AM   #28
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

So, spend some time on refactorin, still not through but the current approach makes it easier.

You can now change the audiofeatures relatively easy. So just go to the tasks\audioanalysis and change the features array and the function that builds the list. Doesnt matter which information, I will just need a dataframe at the end that contains numerical data, size doesn`t matter for the rest of the application as the indexing is now done more dynamically.

Interested in feedback. I hope I can work next week on the tagging part. Extraction of preset information from NKFS works, VSTPreset is a bit more bothering, at least for Steinberg presets; but basically it works as well.
dimentorium is offline   Reply With Quote
Old 03-02-2020, 07:11 AM   #29
brummbear
Human being with feelings
 
brummbear's Avatar
 
Join Date: May 2016
Location: out west
Posts: 301
Default

Quote:
Originally Posted by dimentorium View Post
Did you manage to run the scripts on your computer?
Did not have the time yet as I am traveling. Will try this a soon as I am back and find some time.

Quote:
You can now change the audiofeatures relatively easy. So just go to the tasks\audioanalysis and change the features array and the function that builds the list. Doesnt matter which information, I will just need a dataframe at the end that contains numerical data, size doesn`t matter for the rest of the application as the indexing is now done more dynamically.
Thank you, the array will make this convenient during experimentation and even later when maybe a user could choose this via GUI depending on sound material.

Quote:
By the way it would be interesting if users would send in their presets, to build up a large scale open database with synthesizer sounds for creating models and having them available to everyone.
Agree 100%. Here are some preset collections all donated by NKS users:

https://www.dropbox.com/sh/ulf40cicm...JPtAChOwa?dl=0

Many come with audio previews useful as training material. Particularly the ones for the free and high quality synths "Tyrell N6" and "Surge" are nice to play with:

https://www.dropbox.com/sh/ulf40cicm...nav_tracking=1

https://www.dropbox.com/sh/ulf40cicm...nav_tracking=1


The tagging is of course always somewhat subjective, i.e. how people describe sound characteristics in words (other than providing the name of a real instrument).
brummbear is offline   Reply With Quote
Old 03-03-2020, 02:35 PM   #30
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

Thats interesting with the presets! Will download them and experiment a bit with it.

Also another update. Added the basics for a neural network for the 2D visualization and the tagging. Both not yet tested with larger database so no good results. So load your database and then you can click on the buttons for experimenting.

Also added the table view, when you click on an item, all information (files, features, tags, calculated values) will be displayed.

On my machine I had to use tensorflow 2.0, the newer ones did not work.
dimentorium is offline   Reply With Quote
Old 03-03-2020, 10:57 PM   #31
brummbear
Human being with feelings
 
brummbear's Avatar
 
Join Date: May 2016
Location: out west
Posts: 301
Default

so I tried to run the scripts.... Python 3.8 seems not yet supported by tensorflow => not possible to install the required packages. Python 3.7 works but pip throws the following error:

"ERROR: tensorflow-gpu 2.1.0 has requirement tensorboard<2.2.0,>=2.1.0, but you'll have tensorboard 2.0.2 which is incompatible."

Running the scripts with Python 3.7 did work. The main script throws the following error (I have a Nvidia GTX1060 with latest drivers in this machine):

"Using TensorFlow backend.
2020-03-03 21:46:21.120888: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'cudart64_101.dll'; dlerror: cudart64_101.dll not found
2020-03-03 21:46:21.126184: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine."

Anyhow, the analysis and TSNE map works and showed really good clustering with the 1100 audio previews that I used to test (for Helm synth). NN map seemed to be useless (but that might be because the nksf files did not contain tags?). Pressing the Tagging button crashes the script. Need to play around with this a bit more once I find more time :-)

One problem that I found already with your previous exe version is that the playback of all audio files when clicking on the points in the map is heavily distorted. All my test files are ogg vorbis encoded, maybe I should re-test with wavs... The ogg vorbis files play back MUCH louder and DISTORTED when clicked in the map compared to playing back these files in e.g. foobar2000 or any other player. Have you noticed that too?

Thanks again for all your hard work!
brummbear is offline   Reply With Quote
Old 03-05-2020, 01:02 AM   #32
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

Soo some things I have to guess.

- Python 3.8 is not possible as Keras and Tensorflow are not available. My recommendation (and what I use) is 3.7.6. Best way have a fresh install or virtual environment and use the requirements file in the repository for installing same packages as I use
- I do not have the CUDA or GPU packages installed, I guess there is some mismatch between the libs
- The NN map is getting better, but still not good. Refined it a bit and read some articles
- Tagging works when the tag appears often, then it can be guessed. But trained it yesterday with a small library. Will try the next days with the large library (35000 presets)
- The ogg problem is on two of my machines, but not on my dev computer the same. I tried to lower the level, but it still persists. I guess this is something on samplerate or so. Will need to check

Did yesterday an Update to make the system more stable and faster. This week is probably more on the neural network stuff. Its getting interesting.

If someone needs the exe, just send me a message or drop a line here, then I will try to rebuild it again.

Thanks for your feedback
dimentorium is offline   Reply With Quote
Old 03-10-2020, 02:47 PM   #33
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

Soo still tweaking on my neural network for tagging preview sounds. currently I can estimate 139 categories from Halion with an accuracy of ~80%. Looks interesting and will try to integrate this in the next days, so you can run this with a precompiled model.

On the other hand I am working with a friend on rendering the Omnisphere 2 library to presets and previews that I can also use for tagging and feature extraction. Its a bit annoying that every vendor is making their custom way how to handle presets and no one is sticking to the old VST api for switching presets (not even Steinberg). Looks like my Auto Gui will have to run for some days...
dimentorium is offline   Reply With Quote
Old 03-10-2020, 04:10 PM   #34
brummbear
Human being with feelings
 
brummbear's Avatar
 
Join Date: May 2016
Location: out west
Posts: 301
Default

Quote:
Originally Posted by dimentorium View Post
Its a bit annoying that every vendor is making their custom way how to handle presets and no one is sticking to the old VST api for switching presets (not even Steinberg). Looks like my Auto Gui will have to run for some days...
You may save some time if you use the earlier linked preset magician (https://presetmagician.com/changelog) to extract presets incl audio previews. It comes with both standard VST api extraction incl fxb's as well as vendor specific handlers (incl Omnisphere). Check out the changelog, quite a few already dealt with. Obviously, the problem still remains in the long run, i.e. no standard way of extracting presets from many current and future instruments...
brummbear is offline   Reply With Quote
Old 03-11-2020, 05:56 AM   #35
TonE
Human being with feelings
 
Join Date: Feb 2009
Location: Reaper HAS send control via midi !!!
Posts: 4,031
Default

Quote:
Originally Posted by dimentorium View Post
Its a bit annoying that every vendor is making their custom way how to handle presets and no one is sticking to the old VST api for switching presets (not even Steinberg). Looks like my Auto Gui will have to run for some days...
Then do not use their shit.
TonE is offline   Reply With Quote
Old 03-12-2020, 12:23 AM   #36
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

Hi brummbear,
interesting, that Preset Magician has gone Open source, that helps a lot!
Currently converting 45000 Omnisphere Presets to NKS and then putting them in the database for learning.

Will take 2-3 days for the conversion :-D

But that might get interesting
dimentorium is offline   Reply With Quote
Old 03-16-2020, 08:48 AM   #37
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default

So next version is up, with some fun changes.

You can now automatically tag the presets with a prebuilt model. The model is built via Halion and it does not (yet) incorporate the tags from other libraries. Also the prediction is around 80% but I am working on this.

So simply load your database and click the "Tagging" Button. Then when you click on a preset you will see in the list on the right the estimated tags.

Also finished the Omnisphere Database with 45k Presets and some other Steinberg VSTis from Absolute. Overall I should now hav >50k Presets that I want to use in the future for building the tag model
dimentorium is offline   Reply With Quote
Old 04-14-2020, 11:54 PM   #38
dimentorium
Human being with feelings
 
Join Date: Jan 2020
Posts: 76
Default New concept

Hi there,

took some time off and thought about the concept.
I think I went a bit in the wrong direction so I started new and build a very small tool for storing and saving presets from VST instruments. Maybe this is a better starting point.
Currently you can:
- Save and load a binary database
- Save and Load presets from your instruments

I hope th link works...
https://imgur.com/a/S9ckEqI

Last edited by dimentorium; 04-15-2020 at 12:13 AM.
dimentorium is offline   Reply With Quote
Old 04-15-2020, 03:46 AM   #39
Reflected
Human being with feelings
 
Reflected's Avatar
 
Join Date: Jul 2009
Posts: 3,294
Default

wow!
this is super interesting
Reflected is offline   Reply With Quote
Old 04-15-2020, 03:58 AM   #40
Reflected
Human being with feelings
 
Reflected's Avatar
 
Join Date: Jul 2009
Posts: 3,294
Default

dimentorium,

is there a beta version?
Reflected is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 09:39 AM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.