Go Back   Cockos Incorporated Forums > REAPER Forums > REAPER Feature Requests

Reply
 
Thread Tools Display Modes
Old 06-07-2016, 08:59 PM   #1
cmacsound
Human being with feelings
 
Join Date: Jan 2012
Posts: 38
Default Wishing for text-based audio & video editing feature

First feature request:

I work often with transcript-based material (for documentaries) and hope that one day REAPER could be configured to edit an interview via a text window, so that the corresponding audio items would follow the changes made in text.

I suppose the first step would be to have a phonetic text-align feature so that a transcript could be "imported" or overlaid on top of media items that contain its dialog. Then, one hopefully could edit some text to build a narrative, then perfect the timings and mix it!

This would save me months (maybe years) of my, and hopefully countless others' life.

Some big commercial, examples out there of something like this are ScriptSync, but something tells me that REAPER could do it way better, not to mention more stable...

Thanks for reading this.
cmacsound is offline   Reply With Quote
Old 06-07-2016, 09:44 PM   #2
semiquaver
Human being with feelings
 
Join Date: Jun 2008
Posts: 4,913
Default

this is a very interesting idea and could be implemented as a script I would imagine. Or a plugin.
semiquaver is offline   Reply With Quote
Old 06-08-2016, 09:11 PM   #3
cmacsound
Human being with feelings
 
Join Date: Jan 2012
Posts: 38
Default

Yes indeed! If only I knew how to code such a tool... !

Perhaps a JS plugin with a simple text interface, in conjunction using Gentle would be a step in the right direction:

https://github.com/lowerquality/gentle
cmacsound is offline   Reply With Quote
Old 06-09-2016, 01:08 AM   #4
planetnine
Human being with feelings
 
planetnine's Avatar
 
Join Date: Oct 2007
Location: Lincoln, UK
Posts: 7,585
Default

What editing do you want to do with this tool, specifically?

Can you give a mockup of an instruction set you might want to apply to an audio item? It might be possible in script.



>
planetnine is offline   Reply With Quote
Old 06-09-2016, 12:05 PM   #5
serr
Human being with feelings
 
Join Date: Sep 2010
Posts: 8,292
Default

So... Take a recording of speech. Edit it as text. Push a button and presto-changeo you have a seamless edit of the speech audio with no artifacts?

There are some American TV "news" stations that would probably pay big money for that plugin!
serr is offline   Reply With Quote
Old 06-09-2016, 12:20 PM   #6
cmacsound
Human being with feelings
 
Join Date: Jan 2012
Posts: 38
Default

Yes of course. So glad you think this may be possible to script.

Wishful workflow:

• Import multitrack or stereo mix interview into REAPER.
• Import transcribed interview dialog text into a REAPER JS script for automatic phonetic-syllable alignment, perhaps with the help of Gentle or something similar.

Here's where the magic would come in, after the phonetic alignment of the imported audio dialog and text:

• In a REAPER JS script text editor GUI, a user can edit, rearrange and discard entire sentences and words within the transcript just like one would edit text in a text editor AND have the media items automatically split and rearrange themselves in the TCP timeline, in a way that follows the newly edited structure of the text. (Essentially linking the media item audio to the transcript and vice versa).

• Close/hide the JS transcript editor window and make fine-tune media item edits manually in order to prep for a final mix.

• Reap huge success.

Ideally this script would be inserted an a parent folder track which contains the multitrack of the interview, i.e. host track, guest 1, guest 2, etc. embedded within the parent folder track. I think this way the user could have the ability to edit the script non-destructively, and make further changes in the storyline as the plot develops.

I see this as something that would be a huge game changer for all those producing stories and documentaries from scratch, both in audio and video.

Hope this helps. Please don't hesitate to ask if more clarification is needed!

Last edited by cmacsound; 06-09-2016 at 10:14 PM.
cmacsound is offline   Reply With Quote
Old 06-09-2016, 03:19 PM   #7
planetnine
Human being with feelings
 
planetnine's Avatar
 
Join Date: Oct 2007
Location: Lincoln, UK
Posts: 7,585
Default

I'm following now.

X-Raym is your guy for subtitle actions and scripts, perhaps some of his stuff could be used for extracting edit-point data from the actual wording.

Personally I'd go for highlighting words in a script window and them being correspondingly highlighted in the items themselves for manual editing -you can tell that I don't trust automatic editing, can't you..?



>
planetnine is offline   Reply With Quote
Old 06-09-2016, 04:55 PM   #8
semiquaver
Human being with feelings
 
Join Date: Jun 2008
Posts: 4,913
Default

you would certainly have to go back through all the edits but it would be great to have the script rough it out for you!
semiquaver is offline   Reply With Quote
Old 06-09-2016, 05:42 PM   #9
X-Raym
Human being with feelings
 
X-Raym's Avatar
 
Join Date: Apr 2013
Location: France
Posts: 6,041
Default

@cmacsound
That's an advanced request ^^

Meanwhile, I think you can consider just having your subtitle track with text item and make edit with Ripple Edit All Tracks ative,



Simple and easy solution. Not exactly what is asked, but it can help anyway I guess.
FOr reordering, set edit cursor at the begining of an item, select other items and make your copy paste with ripple active.

ALso, you can split item from first track according to items on track two and group items by columns (I have two scripts for that).
X-Raym is offline   Reply With Quote
Old 06-09-2016, 10:47 PM   #10
cmacsound
Human being with feelings
 
Join Date: Jan 2012
Posts: 38
Default

@X-Raym,

Yes, I realize this is an advanced request, and understand if I'm "shooting the moon" a little in asking for it...

I fully comprehend the text item based ripple edit idea, and thank you for the demonstration reminder. I suppose I'm looking for something that can handle a LOT more audio/text quickly to build a rough outline for script development.

And of course I'd be happy to donate to such a noble development!

@semiquaver,

Of course manual fine-tuning would always be necessary. But if up against countless hours of tape I can definitely see this option to be a HUGE advantage in building narratives and sketching out stories.
cmacsound is offline   Reply With Quote
Old 06-10-2016, 01:33 AM   #11
planetnine
Human being with feelings
 
planetnine's Avatar
 
Join Date: Oct 2007
Location: Lincoln, UK
Posts: 7,585
Default

I think using Raymonds subtitles to at least annotate your audio would be a start. Getting that workflow workable will be essential, whatever level you take your ideas to in the future.

Shooting for the moon is admirable, buts let's build the rocket first.

Can we get a speech recognition app to help with this..?



>
planetnine is offline   Reply With Quote
Old 06-10-2016, 07:45 AM   #12
cmacsound
Human being with feelings
 
Join Date: Jan 2012
Posts: 38
Default

Yes, agreed.

I have my audio annotated (that's the "easy" part thanks to professional transcriptions. I also discovered that YouTube is a decent .srt aligner. For instance, If you upload a video and a transcript text, YouTube aligns all that up and lets you download a .srt file that we can import into REAPER which saves a bunch of time.

Here's what that can look like:


The next step is to move these text items' notes over to the media items' notes, but I can't seem to find a way to do that quickly and easily in a script just yet... ?
cmacsound is offline   Reply With Quote
Old 06-10-2016, 08:14 AM   #13
X-Raym
Human being with feelings
 
X-Raym's Avatar
 
Join Date: Apr 2013
Location: France
Posts: 6,041
Default

@cmacsound
PM sent :P
X-Raym is offline   Reply With Quote
Old 06-24-2016, 06:45 AM   #14
heda
Human being with feelings
 
heda's Avatar
 
Join Date: Jun 2012
Location: Spain
Posts: 5,435
Default

Hi,
I've seen that Gentle is in Python. I don't know python. Everything could be possible. But sometimes it can be very difficult to do it. I'm not good enough to code something like this.
But it is nice that you can do at least the srt workflow with youtube.
heda is offline   Reply With Quote
Old 06-24-2016, 09:06 AM   #15
analogexplosions
Human being with feelings
 
analogexplosions's Avatar
 
Join Date: May 2011
Location: Brooklyn
Posts: 338
Default

I'll be following this closely.
__________________
www.dungeonbeach.com
analogexplosions is offline   Reply With Quote
Old 07-04-2016, 09:49 AM   #16
cmacsound
Human being with feelings
 
Join Date: Jan 2012
Posts: 38
Default

Thanks everyone for chiming in on this thread. Special thanks to X-Raym and HeDa for their very useful text note scripts!

Will continue to research this request in the coming months. For now, here's a site that is also trying to do this very thing, albeit with video on a web interface: http://hyperaud.io/
cmacsound is offline   Reply With Quote
Old 07-19-2016, 01:13 PM   #17
cmacsound
Human being with feelings
 
Join Date: Jan 2012
Posts: 38
Default

A recent REAPER update appears to already help moving things around in the project timelines a little.

Make a timeline selection, and right-click on the timeline. Some new options are available:

– Crop project to selection
– Insert empty space in selection
– Remove contents of selection (moving later items)

http://reaperblog.net/2016/07/quick-...removing-time/
cmacsound is offline   Reply With Quote
Old 09-20-2019, 04:48 AM   #18
X-Raym
Human being with feelings
 
X-Raym's Avatar
 
Join Date: Apr 2013
Location: France
Posts: 6,041
Default

@cmacsound


I found a text based editing software :



https://www.descript.com/


It has export to EDL, AAF, Premiere Pro etc... You could then do post prod in REAPER
X-Raym is offline   Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT -7. The time now is 10:12 PM.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.