|
|
|
09-19-2016, 04:39 PM
|
#1
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
Q: Parse CSV file with JSFX
Hi,
I am a bit confused that string format and function in JSFX.
imagine a file like
The seperator - can be replace with any other separator.
Here is the code for sampling a file (from MIDI Pattern/Scale Variation Generator [IXix])
Code:
@slider
seqFile != slider8 ?
(
seqFile = slider8;
seqSize = 0;
fileHandle = file_open(slider8);
fileHandle > 0 && file_text(fileHandle) ?
(
while
(
file_var(fileHandle,seq[seqSize]);
file_avail(fileHandle) ? seqSize += 1;
);
file_close(fileHandle);
);
);
You can then access a line with seq[x] where x is the number of the line.
But how will you return two variables from that line ? How would you parse the "45-50" into two int variable ?
I only succeed to get int when there is only one number per line, but not two with a sperator.
Thanks for your help !
Last edited by X-Raym; 09-19-2016 at 07:30 PM.
|
|
|
09-19-2016, 07:01 PM
|
#2
|
Human being with feelings
Join Date: Mar 2007
Location: I'm in a barn
Posts: 4,467
|
The internal parsing that IXIX's code is doing is tied specifically to sliders and ignores non numbers, and con only read a single variable.
however, you could do
and manipulate it mathematically to parse the variables.
so you'd have to use a text editor to convert " - " into "."
You can use file_string(handle,str) to get the text, however, this only reads the first line - it should be able to do newlines, might be some encoding issue, might be a bug. will AskJF.
*asked justin and the great oracle answered see post #11
Last edited by James HE; 09-20-2016 at 05:22 PM.
|
|
|
09-19-2016, 07:44 PM
|
#3
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
@JamesHE
thank you james for the precision, it helps,
I missed the file_string function, is the second parameter supposed to be the line number in "read" mod ?
But yes indeed, a solution without match/string function can work for my particular case.
The number tricks is a good idea, not very difficult to implement (just math floor for the first value)
I will change the - to . inside the script which generates this CSV.
Stay tuned for the script, should came out tomorrow :P
Last edited by X-Raym; 09-20-2016 at 04:03 AM.
|
|
|
09-19-2016, 10:40 PM
|
#4
|
Human being with feelings
Join Date: Oct 2007
Location: Lincoln, UK
Posts: 7,926
|
What about a comma?
Isn't that what the "C" is in CSV? ( comma separated values)
>
|
|
|
09-20-2016, 01:30 AM
|
#5
|
Human being with feelings
Join Date: May 2006
Location: Surrey, UK
Posts: 19,677
|
Wouldn't the match() function be the one to use?
I would post the answer but have never really understood how to define the matching pattern
__________________
DarkStar ... interesting, if true. . . . Inspired by ...
|
|
|
09-20-2016, 03:01 AM
|
#6
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
@planetnine
Actuallly, it is for "Character", it is very often a space or a a tab :P
But the problem is more about variable value than line formatting, I can change it without problem.
@DarkStar
spk77 and gofer already made a example of match function used in eel,
ReaScripts-Templates/gofer-spk77-Functions - User inputs fields.eel at master · ReaTeam/ReaScripts-Templates
but for some reason it didn't work with the way the file is parsed using the code I show in post 1.
JamesHE explained why: this parsing method doesn't support strings
@all
If anyone find a workable solution involving more columns (without math tricks), I'm sure it could be very useful.
Thanks for your suggestions !
|
|
|
09-20-2016, 04:37 AM
|
#7
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
Hm the math trick is a bit harde rthan expected,
hard to differentiate between
vel = 1
vel = 10
vel = 100
if it is store as a decimal 0.1, 0.10, 0.100... it is all the same in math functions. :/
Can't we return string from file based on a line number ?
EDIT: oh wait, no problem in fact, as I can change output format from my MIDI to CSV script, I can make it encode velocity as 0.xxx number instead of just 0.vel :P
Last edited by X-Raym; 09-20-2016 at 04:57 AM.
|
|
|
09-20-2016, 09:42 AM
|
#8
|
Human being with feelings
Join Date: Mar 2007
Location: I'm in a barn
Posts: 4,467
|
Actually, file_string() can read multiple lines, you just need to call it multiple times.
|
|
|
09-20-2016, 09:46 AM
|
#9
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
@JamesHE
Cool !
Do you have any working code snippet about how we can loop in all lines of a file ?
|
|
|
09-20-2016, 04:02 PM
|
#10
|
Human being with feelings
Join Date: Mar 2007
Location: I'm in a barn
Posts: 4,467
|
Quote:
Originally Posted by X-Raym
@JamesHE
Cool !
Do you have any working code snippet about how we can loop in all lines of a file ?
|
writing the functions now
back in a bit.
|
|
|
09-20-2016, 05:08 PM
|
#11
|
Human being with feelings
Join Date: Mar 2007
Location: I'm in a barn
Posts: 4,467
|
wow, what a hard challenge it can be to translate what the jsfx help is telling you ha ha ha. It's funny (sometimes beautiful) how simple it is once you get it.
Code:
function file_to_numbered_strings(file,offset)local(handle)
//offset= starting string number
//returns number of lines read
(
handle=file_open(file);
file_string(handle,offset);
while( strlen(offset) && offset < 1023 )
( offset+=1;
file_string(handle,offset);
);
file_close(file);
offset;
);
so this breaks up each line in a txt file into numbered strings. From there, matching into variables should be fine.
Offset will generally be 0, but you never know - you might want to to chain these together for different files - just pass the returned offset to the next function call.
Note that there are 1024 slots for numbered strings, so I made this stop reading at that limit (cause I'm not sure what happens then). If you'd need to read war and Peace with JS, you could start stuffing the strings into memory - or to even serialize them, but for now, lets just keep it simple and assume 1024 lines is enough.
A different function could also parse the lines into a #NamedBigString - possibly adding newlines or some other character (or not) - and then you just get your variables from it. I would do that if I needed to serialize the string - or even if I needed to make sure some other function doesn't overwrite the numbered strings and i needed to preserve the slots (or pass it to another function later in the code - cause things get weird if you don't)
If you need anything else, let me know. I have EEL string functions that can chop up complex strings (i.e a track chunk) like a sou chef chopping an onion. (peek into my ReaLComps script files (the library that comes with it) if you want - most are in there)
Last edited by James HE; 09-20-2016 at 05:26 PM.
|
|
|
09-20-2016, 11:17 PM
|
#12
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
@JamesHE
Very cool james !
I put this snippet on ReaTeam template repo:
ReaScripts-Templates/JamesHE_Parse file lines into string.jsfx at master · ReaTeam/ReaScripts-Templates
I'm sure other script may find it useful.
Quote:
what a hard challenge it can be to translate what the jsfx help is telling you
|
Indeed, it is usually where I am stuck haha :P
Quote:
If you need anything else, let me know.
|
Thanks for you offer
if you are ok and this is not too much to ask (you already made a lot with your function), making a complete functional demo script it could be very helpful
Like having a the script looking for files from a slider in the a Data subfolder (or in project folder, if this is possible - I didn't find how), and a way to call/debug one particular line, and optionally to make a pattern search like if each lines is a two column CSV.
For the absolute beginner, all details can matter, like where to declare the function within the script, where to call it, how to name the file in the call (does it require to put the .txt extension or not)...
Are you in ? :P
|
|
|
09-21-2016, 06:49 PM
|
#13
|
Human being with feelings
Join Date: Mar 2007
Location: I'm in a barn
Posts: 4,467
|
Quote:
Originally Posted by X-Raym
@JamesHE
Like having a the script looking for files from a slider in the a Data subfolder (or in project folder, if this is possible - I didn't find how),
|
I think to use text files with a slider, they either need to be in the exact same folder as the JS plugin (I think REAPER actually looks there first) or in the DATA directory.
I just always use the DATA directory cause I know where to find it
Here is a working JS plug example that just displays the context of the text files. The user will have to provide the files of course
Code:
desc:.txt Files examples > by James HE
slider1:/text files:none:Text File
@init
//FUNCTIONS for reading .txt files
//offset = starting string number
//returns number of lines read
function file_to_numbered_strings(file,offset)local(handle)
//this version can be used when you define the file via ~ filename:0,sometext.txt at the beginning of your code
//use slider version if you want to pick from text files in a folder
(
handle=file_open(file);
file_string(handle,offset);
while( strlen(offset) && offset < 1023 )
( offset+=1;
file_string(handle,offset);
);
file_close(file);
offset;
);
function slider_file_to_numbered_strings(slidernumber,offset)local(handle,lastvalue,lines)
//USE ONLY THE NUMBER OF THE SLIDER FOR "slidernumber" - do not use "sliderx"
(
slider(slidernumber) != lastvalue ? reload=0;
!reload ? (
handle=file_open(slider(slidernumber));
file_string(handle,offset);
while( strlen(offset) && offset < 1023 )
( offset+=1;
file_string(handle,offset);
);
file_close(file);
lines=offset;
lastvalue=slider(slidernumber);
reload=1;
);
lines;
);
@slider
lines=slider_file_to_numbered_strings(1,offset);
@gfx 650 450
//some guidance
gfx_setfont(1, Ariel,14);
gfx_x=40; gfx_y=10;
gfx_r=gfx_g=gfx_b=1;
gfx_drawstr("Create a folder in your REAPER Data directory named \"text files\", then create or put some .txt files in the folder.
The text within those files will be displayed below when selected by the slider");
//display file
gfx_setfont(2, Ariel,18);
gfx_r=1;gfx_g=gfx_b=0;
xx=30;
yy=55;
gfx_x=xx;gfx_y=yy;
i=0;
loop ( lines,
gfx_drawstr(i);
gfx_x=xx;
gfx_y+=gfx_texth+3;
i+=1;
);
*When working with .txt files like this and something seems to disappear, the file might not have been closed properly. OFFLINE the plugin and bring it back if something goes awry.
|
|
|
09-22-2016, 01:36 AM
|
#14
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
Excellent !
I updated the code snippet on GitHub Thanks !
As I can see, to display the line of text, we only use gfx_drawstr(i);.
So it seems that the strings are loaded in some kind of buffer where we only need to pass the line number as argument of the draw function to display them in GFX.
But how could we access a particular line without drawing it in GFX (to store it in another variable for exemle). ?
|
|
|
09-22-2016, 04:27 AM
|
#15
|
Human being with feelings
Join Date: Mar 2007
Location: I'm in a barn
Posts: 4,467
|
Quote:
Originally Posted by X-Raym
Excellent !
I updated the code snippet on GitHub Thanks !
As I can see, to display the line of text, we only use gfx_drawstr(i);.
So it seems that the strings are loaded in some kind of buffer where we only need to pass the line number as argument of the draw function to display them in GFX.
But how could we access a particular line without drawing it in GFX (to store it in another variable for exemle). ?
|
each line is stored in the numbered string slots. To get a variable from it, you use match() to get what you are looking from out of that numbered string.
|
|
|
09-22-2016, 05:11 AM
|
#16
|
Human being with feelings
Join Date: May 2006
Location: Surrey, UK
Posts: 19,677
|
^^^
That's what I hinted at
Code:
str_11 = "45-50-53"; match("%d%d%d", str_11, v11a, v11b, v11c);
str_12 = "40 51 55"; match("%d %d %d", str_12, v12a, v12b, v12c);
str_13 = "68,98,108"; match("%d,%d,%d", str_13, v13a, v13b, v13c);
gfx_x =20; gfx_y =20;
gfx_r=1.00; gfx_g=0.80; gfx_b =0.00; gfx_a = 0.80;
gfx_x =20; gfx_y +=15; gfx_printf("%3d %3d %3d", v11a, v11b, v11c);
gfx_x =20; gfx_y +=15; gfx_printf("%3d %3d %3d", v12a, v12b, v12c);
gfx_x =20; gfx_y +=15; gfx_printf("%3d %3d %3d", v13a, v13b, v13c);
Note that the '-' character is parsed as negating the following digits, not as a separator.
__________________
DarkStar ... interesting, if true. . . . Inspired by ...
|
|
|
09-22-2016, 05:42 AM
|
#17
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
thanks both of you, I get it :P
Code:
//display file
gfx_setfont(2, Ariel,18);
gfx_r=1;gfx_g=gfx_b=0;
xx=30;
yy=55;
gfx_x=xx;gfx_y=yy;
i=0;
loop ( lines,
match("%d-%d-%d", i, v11a, v11b, v11c); // Parse line i
sprintf(str, "%d", v11a); // Convert to string for display in GFX
gfx_drawstr(str); // Draw line str
gfx_x=xx;
gfx_y+=gfx_texth+3;
i+=1;
);
each line get parsed into several variable,
and as extra feature, one of these is converted to string and displayed in GFX.
|
|
|
06-25-2019, 04:07 PM
|
#18
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
Hi,
Is it possible to parse CSV file like
etc...
so that the first number become index of an array containing the string as value ?
I tried a lot of things including,
Code:
map = 100;
//Load map
mapFile != slider30 | 0 || reload == 1 ?
(
reload = 0;
mapSize = 0;
mapFile = slider30;
fileHandle = file_open(slider30);
fileHandle > 0 && file_text(fileHandle) ?
(
while
(
file_string(fileHandle, str);
match("%d %s", str, d, #s);
map[d] = #s;
file_avail(fileHandle) ? mapSize += 1;
);
file_close(fileHandle);
);
);
Without success :S
|
|
|
06-26-2019, 01:50 AM
|
#19
|
Human being with feelings
Join Date: Jul 2008
Location: The Netherlands
Posts: 3,646
|
Something like this should probably work:
Code:
undef = 0;
strcpy(undef, "[Undef]");
memset(map, undef, 128);
handle = file_open(slider30);
handle >= 0 ? (
file_text(handle) ? (
str = undef + 1;
while(
file_string(handle, #line);
file_avail(handle) ? (
// You could probably deal with line-endings more efficiently...
(match("%d %s\r\n", #line, note, str) ||
match("%d %s\n", #line, note, str)) &&
note >= 0 && note < 128 ? (
map[note] = str;
str += 1;
);
1; // while
);
);
);
file_close(handle);
);
|
|
|
06-26-2019, 04:08 AM
|
#20
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
@Tale
Thx for your help !
Unfortunatly this face the same problem has I had when testing: either only the first or last line works.
Here is a full exemple:
CSV:
Code:
37 banjo.png
36 cowbell.png
JSFX:
Code:
desc:Parse MIDI Note Name File
slider2:36<0,127,1>Base Note
slider30:/track_icons:none:Text File
////////////////////////////////////////////////////////////////////////////////
@init
////////////////////////////////////////////////////////////////////////////////
@slider
undef = 0;
strcpy(undef, "[Undef]");
memset(map, undef, 128);
handle = file_open(slider30);
handle >= 0 ? (
file_text(handle) ? (
str = undef + 1;
while(
file_string(handle, #line);
file_avail(handle) ? (
// You could probably deal with line-endings more efficiently...
(match("%d %s\r\n", #line, note, str) ||
match("%d %s\n", #line, note, str)) &&
note >= 0 && note < 128 ? (
map[note] = str;
str += 1;
);
1; // while
);
);
);
file_close(handle);
);
@gfx 100 32
gfx_setfont(1, "Arial", 20);
gfx_r=1; gfx_b=1; gfx_g=1; gfx_a=1;
gfx_x=gfx_y=5;
gfx_drawstr(map[slider2]);
In this case, 37 do return "banjo.Png" as expected, but 36 return "[Undef]"...
Any idea ?
|
|
|
06-26-2019, 04:10 AM
|
#21
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
Oh I know what is wrong,
The last line isn't return for some reason, I tested with more entries. I wonder how to fix it.
|
|
|
06-26-2019, 04:23 AM
|
#22
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
This seems to work;
Code:
desc:Parse MIDI Note Name File
slider2:36<0,127,1>Base Note
slider30:/track_icons:none:Text File
////////////////////////////////////////////////////////////////////////////////
@init
////////////////////////////////////////////////////////////////////////////////
@slider
undef = 0;
strcpy(undef, "[Undef]");
memset(map, undef, 128);
handle = file_open(slider30);
handle >= 0 ? (
file_text(handle) ? (
str = undef + 1;
while(
file_string(handle, #line);
// You could probably deal with line-endings more efficiently...
match("%d %s", #line, note, str) &&
note >= 0 && note < 128 ? (
map[note] = str;
str += 1;
);
file_avail(handle) ;
);
);
file_close(handle);
);
@gfx 100 32
gfx_setfont(1, "Arial", 20);
gfx_r=1; gfx_b=1; gfx_g=1; gfx_a=1;
gfx_x=gfx_y=5;
gfx_drawstr(map[slider2]);
Differences:
- remove new line character from patterns
- while loop condition is not file_avail
Do you see anything wrong with this method ?
|
|
|
06-26-2019, 04:46 AM
|
#23
|
Human being with feelings
Join Date: Jul 2008
Location: The Netherlands
Posts: 3,646
|
Quote:
Originally Posted by X-Raym
Do you see anything wrong with this method ?
|
Well, if you don't remove the newline chars now, then you will likely still have to deal with them at some point (especially because there could de different versions, depending on source file, OS, etc.).
And about file_avail() and text files, this is what the JSFX documentation says:
Quote:
Note that file_avail() should be called to check for EOF after each read, and if it returns 0, the last file_var() should be ignored.
|
It doesn't mention file_string(), so maybe it works different from file_var() in this regard? Or maybe it depends on whether the last line has newline chars or not?
|
|
|
06-26-2019, 05:13 AM
|
#24
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
Quote:
Well, if you don't remove the newline chars now, then you will likely still have to deal with them at some point (especially because there could de different versions, depending on source file, OS, etc.).
|
As I'm not sure file_string actually return this character, it seems to check file line after line (so stripping new line character).
Anyway, it works on windows like that, so if it works on mac too, it will be a valid solution
Have you tried it on Mac ?
Quote:
It doesn't mention file_string(), so maybe it works different from file_var() in this regard? Or maybe it depends on whether the last line has newline chars or not?
|
Not sure what happen there indeed, but from the end user point of view, having to know that he have to leave an empty line at the end of the file is a necessary information (it will be better to not have to remember such details).
Thx for the assistance
|
|
|
06-26-2019, 06:59 AM
|
#25
|
Human being with feelings
Join Date: Jul 2008
Location: The Netherlands
Posts: 3,646
|
Quote:
Originally Posted by X-Raym
As I'm not sure file_string actually return this character, it seems to check file line after line (so stripping new line character).
Anyway, it works on windows like that, so if it works on mac too, it will be a valid solution
|
Are you sure? Because here "\r\n" or "\n" are not stripped. Note that simply doing gfx_drawstr() will mask the issue, but if you replace it with this then you will probably see what I mean:
Code:
gfx_drawchar('<');
gfx_drawstr(map[slider2]);
gfx_drawchar('>');
Quote:
Originally Posted by X-Raym
Have you tried it on Mac ?
|
I just did, and it seems to work the same as on Windows, i.e. it also doesn't strip (or convert) "\r\n" or "\n". But at least it does work consistent across Windows and macOS (and probably also Linux).
--
I think you are right about file_string() and file_avail(), so maybe this would make more sense then:
Code:
while(file_avail(handle)) (
file_string(handle, #line);
...
);
|
|
|
06-26-2019, 07:14 AM
|
#26
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
Ok for stripping, I can still handle after parsing
Cheers !
|
|
|
06-26-2019, 07:21 AM
|
#27
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
Quote:
match("%d %s*", #line, note, str)
|
This seems to do the trick !
|
|
|
06-26-2019, 07:22 AM
|
#28
|
Human being with feelings
Join Date: Jun 2013
Location: Krefeld, Germany
Posts: 14,690
|
Indeed file_string() works fine for reading lines. See ReaPack->MIDI CCTable.
-Michael
|
|
|
11-07-2023, 04:00 AM
|
#29
|
Human being with feelings
Join Date: Jun 2023
Posts: 3
|
hi X-Raym !
thanks for the great plugin! it's really more convenient than anything else I've found to solve this task.
I used it with success until recently.
but I had to reinstall win, and at the same time update the reaper to 7.
and after that there were problems with the plugin.
here's what they are:
from time to time the plugin does not replace the input note with the assigned output one,
Accordingly, the same note that entered is passed to the output of the plugin.
This mainly occurs when multiple notes are played at the same time.
What could be the reason? What steps can I try to take to resolve this error?
thanks again!
|
|
|
11-07-2023, 04:14 AM
|
#30
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
@shpako
Hi !
I'm not sure to get what script you are referring top as this thread is a genric technical question about JSFX, not related to one particular script.
Though I dont have many scripts with this feature, but some of them break, it's too early to say it's because of the file parsing part.
So let's go private to not be off topic, and send me a .rpp with a track, the JSFX, the mini item, the CSV.
The bug should be reproductible (and if possible obvious) once I press Play.
You can double this with a video of the issue with explainations.
Thx !
|
|
|
11-07-2023, 04:24 AM
|
#31
|
Human being with feelings
Join Date: Jun 2023
Posts: 3
|
ok. I did
Last edited by shpako; 11-07-2023 at 12:21 PM.
|
|
|
Thread Tools |
|
Display Modes |
Linear Mode
|
Posting Rules
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is Off
|
|
|
All times are GMT -7. The time now is 07:55 AM.
|