|
|
|
05-11-2019, 10:44 AM
|
#1
|
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Video scene detection script
Just released a free video scene detection script in the mrlimbic reapack repository.
It's called "mrlimbic_ffprobe scene detect.lua"
You'll need to install ffprobe for it to work. Read the instructions at the top of the script.
To use, you select the track with video items to analyse.
It will create a new track and generate empty items on it to represent the cuts.
It may take a while to detect cuts depending on the size of the video.
Video demo here..
http://vordio.net/wp-content/uploads...ect-Script.m4v
|
|
|
05-11-2019, 12:12 PM
|
#2
|
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
So you really did it? WOW!
I'm curious about how it works.
Greetings
Eli
And many thanks of course.
Edit: Okay, I saw the video. This is just stunning!
(づ。◕‿‿◕。)づ -`ღ´-
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
|
|
|
05-11-2019, 12:26 PM
|
#3
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
The next phase would be to impor t small thumbnails... This could be a usable workarround untill a propper native solution is found !
|
|
|
05-11-2019, 12:33 PM
|
#4
|
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
Originally Posted by Eliseat
So you really did it? WOW!
I'm curious about how it works.
Greetings
Eli
And many thanks of course.
Edit: Okay, I saw the video. This is just stunning!
(づ。◕‿‿◕。)づ -`ღ´-
|
The last attempt used pyscenedetect but that turned out to be such a nightmare to install with so many dependencies that I gave up with that approach and started again.
This attempt instead uses ffprobe which is much easier for people to download and install in one go.
I'll add JPEG thumbnails to it later (using ffmpeg to extract a list of frames).
|
|
|
05-11-2019, 12:38 PM
|
#5
|
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
Works amazing!
And of course I have just some suggestions.
Would it be possible to show a waiting screen? I have the same problem with my mkvtoolnix split script that shows no response if the working process lasts to long. Have actually no idea how to do that, but I will look around for a solution.
It would also be nice to have an auto split version which cuts the source video in Reaper at the right spots.
AND another idea would be to put a video processor in every item fx of a scene with the text overlay preset on, which would display the scene in the video window...
No, I'm just joking! That would be decadent.
Well done!
Eli
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
|
|
|
05-11-2019, 12:42 PM
|
#6
|
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
By the way: Do you know if ffmpeg is also capable of splitting videos without re-encode? Would it be possible to lossless split a video from inside of Reaper and importing the created clips as items on a new track?
Many people were asking for that. And it would be a good motivation for me to get into lua in Reaper.
Edit: Scene images are a great idea. Gives a nice professional touch to it.
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
|
|
|
05-11-2019, 12:52 PM
|
#7
|
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
Originally Posted by Eliseat
By the way: Do you know if ffmpeg is also capable of splitting videos without re-encode? Would it be possible to lossless split a video from inside of Reaper and importing the created clips as items on a new track?
Many people were asking for that. And it would be a good motivation for me to get into lua in Reaper.
|
Yes ffmpeg can split videos without re-encoding (using copy stream parameter). BUT not all codecs can be cut on every frame when copying only. Some can only be cut on keyframes.
If there are keyframes at seconds 1, 2, and 3, but you want to cut at second 1.5, then ffmpeg will wait until second 2 before it can start cutting.
|
|
|
05-11-2019, 01:15 PM
|
#8
|
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
Originally Posted by Eliseat
Would it be possible to show a waiting screen? I have the same problem with my mkvtoolnix split script that shows no response if the working process lasts to long. Have actually no idea how to do that, but I will look around for a solution.
|
The only thing I can think of it to popup the debug message console and print dots to it while checking if the processing by ffprobe has finished. (It checks about 30 times per second until the cut list file arrives)
However there appears to be no way to close the console automatically after. You'd have to close it by hand. So would this be annoying?
|
|
|
05-11-2019, 01:43 PM
|
#9
|
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Here's what popping up the message console looks like for processing a 5 minute video.
Any good?
|
|
|
05-11-2019, 02:28 PM
|
#10
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
@mrlimbic
I would rather prefer a Clear Console with a timecode like "Elapsed time: 30s..." rather than just points :P
Will look a bit more clean.
|
|
|
05-11-2019, 03:12 PM
|
#11
|
Human being with feelings
Join Date: Jun 2009
Location: South, UK
Posts: 14,214
|
whoa!
I put a reminder in my calendar to ask if a script like this was possible and I just came to the site to post a thread when I saw this! lol
Amazing stuff thanks!
I'll try it out when I get time!
Been wanting this for years, will donate if you do donations!
__________________
subproject FRs click here
note: don't search for my pseudonym on the web. The "musicbynumbers" you find is not me or the name I use for my own music.
|
|
|
05-11-2019, 03:54 PM
|
#12
|
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
Quote:
Originally Posted by mrlimbic
Yes ffmpeg can split videos without re-encoding (using copy stream parameter). BUT not all codecs can be cut on every frame when copying only. Some can only be cut on keyframes.
If there are keyframes at seconds 1, 2, and 3, but you want to cut at second 1.5, then ffmpeg will wait until second 2 before it can start cutting.
|
Yes, I know that. I had the same problem with my mkvtoolnix video splitter for vlc. It also cuts only at codec keyframes, what makes sense, since those are the only real frames/pictures in a film while the other frames are only interpolated.
The question is, if Reaper could send the split information from itself and getting the real split information back from ffmpeg. Okay, that sounds a bit confusing. But to get this working right Reaper needs the real split data from ffmpeg to import the real clips as items. Okay, that sounds also confusing.
Lets say I have a video of 10 minutes length. I cut exactly at 30 seconds and at 45 seconds, send this information to ffmpeg which can't cut at these points because the keyframes are at sec. 31 and sec. 47. That means Reaper needs info back how long the clip really is AND needs to place it somehow as item on a new track.
Would that be possible? Theoretically?
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
|
|
|
05-11-2019, 03:55 PM
|
#13
|
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
Originally Posted by musicbynumbers
whoa!
Been wanting this for years, will donate if you do donations!
|
I don't normally do donations but twelfty pounds would be nice.
https://www.youtube.com/watch?v=BNJgy6j9gas
|
|
|
05-11-2019, 03:56 PM
|
#14
|
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
Quote:
Originally Posted by X-Raym
@mrlimbic
I would rather prefer a Clear Console with a timecode like "Elapsed time: 30s..." rather than just points :P
Will look a bit more clean.
|
Do you have an idea of how do that? I mean the dots are at least better than the no response behavior of the DAW.
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
|
|
|
05-11-2019, 04:00 PM
|
#15
|
Human being with feelings
Join Date: Mar 2018
Location: Cologne
Posts: 1,362
|
Quote:
Originally Posted by mrlimbic
Here's what popping up the message console looks like for processing a 5 minute video.
Any good?
|
I find it better than the no response behavior. If someone in the meanwhile clicks accidentally on a window, the system asks for shutdown the "frozen" Reaper. That would be the worst case.
__________________
☆.。.:*・°☆.。.:*・°☆.。.:*・°☆REAPER//✿◔‿◔)°☆.。.:*・°☆.。.:*・°☆
|
|
|
05-11-2019, 04:22 PM
|
#16
|
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
Originally Posted by Eliseat
I find it better than the no response behavior. If someone in the meanwhile clicks accidentally on a window, the system asks for shutdown the "frozen" Reaper. That would be the worst case.
|
You should be able to shut the debugging console window without problems (it's just not normally visible to users). I haven't experienced any crashes. Have you?
|
|
|
05-11-2019, 04:26 PM
|
#17
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
Quote:
Do you have an idea of how do that?
|
Of course,
I also moded the script to bring the Windows compatibility :P
@mrlimbic
For auto closing console:
https://forum.cockos.com/showpost.ph...11&postcount=2
Using JS_reascriptAPI functions for getting the window.
|
|
|
05-11-2019, 05:10 PM
|
#18
|
Human being with feelings
Join Date: Jun 2009
Location: South, UK
Posts: 14,214
|
Quote:
Originally Posted by mrlimbic
|
You know that's based on a cafe/shop in Rottingdean that actually sold precious things (snow globes)! I don't know if it's still there though.
__________________
subproject FRs click here
note: don't search for my pseudonym on the web. The "musicbynumbers" you find is not me or the name I use for my own music.
|
|
|
05-11-2019, 05:15 PM
|
#19
|
Human being with feelings
Join Date: Jul 2009
Posts: 7,568
|
how is a scene change defined and can it be tweaked? I only did a quick test but there were a bunch of missed cuts where there was one static element like text over close and medium shots of same subject, or when there's a fade between totally different scenes.
|
|
|
05-11-2019, 05:22 PM
|
#20
|
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
Originally Posted by EpicSounds
how is a scene change defined and can it be tweaked? I only did a quick test but there were a bunch of missed cuts where there was one static element like text over close and medium shots of same subject, or when there's a fade between totally different scenes.
|
It's basically set to 40% of change in colour space since last frame. I could make that a tweakable parameter. For some videos it may not work well at all not matter what threshold you set. Fades will never work well I think.
|
|
|
05-11-2019, 06:55 PM
|
#21
|
Human being with feelings
Join Date: Sep 2018
Location: China
Posts: 565
|
THAT IS CRAZY! Will it be a free script or not? I'll pay for it!
And a side question here, are you still working on Vordio?
|
|
|
05-11-2019, 07:04 PM
|
#22
|
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
Originally Posted by dsyrock
THAT IS CRAZY! Will it be a free script or not? I'll pay for it!
And a side question here, are you still working on Vordio?
|
This script is free. I don't like charging for something where most of the work is done by another tool (ffprobe here).
Yes Vordio is still going and have been working on big new things coming in Vordio 6. It's just been way more work than I expected and taking a lot longer. It's almost a complete rewrite now. I thought it would be ready around feb but still a bit to do.
So not many updates on Vordio 5 any more as that is now only bug fixes. None of the code is compatible with any of the new V6 stuff.
|
|
|
05-11-2019, 07:16 PM
|
#23
|
Human being with feelings
Join Date: Sep 2018
Location: China
Posts: 565
|
OK, thanks. Back to this script, are you using something like os.execute() to execute ffprobe.exe?
|
|
|
05-11-2019, 07:24 PM
|
#24
|
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
Originally Posted by dsyrock
OK, thanks. Back to this script, are you using something like os.execute() to execute ffprobe.exe?
|
Look at the script code around line 122. Yes for some weird reason what works best to call the executable on mac vs windows is different. I don't know why.
|
|
|
05-11-2019, 07:27 PM
|
#25
|
Human being with feelings
Join Date: Sep 2018
Location: China
Posts: 565
|
Quote:
Originally Posted by mrlimbic
Look at the script code around line 122. Yes for some weird reason what works best to call the executable on mac vs windows is different. I don't know why.
|
Wait, the script has been released? Is it on reapack?
|
|
|
05-12-2019, 12:33 AM
|
#26
|
Human being with feelings
Join Date: Nov 2014
Location: Can Tho - Viet Nam
Posts: 305
|
Thank you
Awesome ! Thank you!
|
|
|
05-12-2019, 08:14 AM
|
#27
|
Human being with feelings
Join Date: Jun 2009
Location: South, UK
Posts: 14,214
|
Quote:
Originally Posted by dsyrock
Wait, the script has been released? Is it on reapack?
|
I think so yes, about to try it myself. Info is in the first post, you need some ffmpeg stuff too (ffprobe I think).
__________________
subproject FRs click here
note: don't search for my pseudonym on the web. The "musicbynumbers" you find is not me or the name I use for my own music.
|
|
|
05-12-2019, 09:31 AM
|
#28
|
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
Originally Posted by musicbynumbers
I think so yes, about to try it myself. Info is in the first post, you need some ffmpeg stuff too (ffprobe I think).
|
Yes it's in the mrlimbic reapack repository.
Currently it only needs ffprobe in the UserPlugins directory (or if it is not found there it will ask you where your ffprobe executable is with a file open dialog). You may want to put it elsewhere. I am currently having video issues (green frames occasionally), so not sure if reaper just doesn't like ffmpeg there, or if it's just a cooincidence.
However, when I add thumbnail generation later it will also require ffmpeg to do that part.
|
|
|
05-12-2019, 10:03 AM
|
#29
|
Human being with feelings
Join Date: Jun 2009
Location: South, UK
Posts: 14,214
|
Quote:
Originally Posted by mrlimbic
Yes it's in the mrlimbic reapack repository.
Currently it only needs ffprobe in the UserPlugins directory (or if it is not found there it will ask you where your ffprobe executable is with a file open dialog). You may want to put it elsewhere. I am currently having video issues (green frames occasionally), so not sure if reaper just doesn't like ffmpeg there, or if it's just a cooincidence.
However, when I add thumbnail generation later it will also require ffmpeg to do that part.
|
good info thanks. Going to check it now. Very happy to hear you're aiming to get thumbnail generation too!
__________________
subproject FRs click here
note: don't search for my pseudonym on the web. The "musicbynumbers" you find is not me or the name I use for my own music.
|
|
|
05-13-2019, 06:45 AM
|
#30
|
Human being with feelings
Join Date: Mar 2011
Location: London
Posts: 1,211
|
This script will immediately be so useful to me!
But unfortunately even a 25s video seems to take forever to analyze (I've been waiting for 5 minutes and it still hasn't finished). Or does the source file length (85 minutes) impact the time it takes for the script to complete its job?
Also, killing the script doesn't seem to kill the ffprobe process.
|
|
|
05-13-2019, 07:06 AM
|
#31
|
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
Originally Posted by swiiscompos
This script will immediately be so useful to me!
But unfortunately even a 25s video seems to take forever to analyze (I've been waiting for 5 minutes and it still hasn't finished). Or does the source file length (85 minutes) impact the time it takes for the script to complete its job?
|
Are you on mac or windows?
ffprobe may actually not be running. I don't think a script can kill a process.
But there is no progress shown yet so hard to know. Next version will show progress. On my laptop it takes 1 second per minute of video. A feature length film took 1:40. If you look in task manager/activity monitor does ffrobe even start?
When running it produces a cut list file in your reaper project folder that ends with .temp, which when complete gets renamed to .txt. What has appeared in your project folder?
The whole source file is analysed. I only tested it with whole files. I've realised that cuts will likely be in the wrong place if video doesn't start at the beginning. The media source in point is not accounted for at the moment.
I'll do another update later today with these two fixes.
|
|
|
05-13-2019, 07:17 AM
|
#32
|
Human being with feelings
Join Date: Mar 2011
Location: London
Posts: 1,211
|
Quote:
Originally Posted by mrlimbic
Are you on mac or windows?
ffprobe may actually not be running. I don't think a script can kill a process.
But there is no progress shown yet so hard to know. Next version will show progress. On my laptop it takes 1 second per minute of video. A feature length film took 1:40. If you look in task manager/activity monitor does ffrobe even start?
When running it produces a cut list file in your reaper project folder that ends with .temp, which when complete gets renamed to .txt. What has appeared in your project folder?
The whole source file is analysed. I only tested it with whole files. I've realised that cuts will likely be in the wrong place if video doesn't start at the beginning. The media source in point is not accounted for at the moment.
I'll do another update later today with these two fixes.
|
I'm on Mac. Ffprobe was still running in the Activity Monitor after I killed the process. Now the good news, I started the script again with the whole video (86 minutes). I waited for a while but it ended up getting all of the scenes right!
|
|
|
05-13-2019, 07:23 AM
|
#33
|
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
Originally Posted by swiiscompos
I'm on Mac. Ffprobe was still running in the Activity Monitor after I killed the process. Now the good news, I started the script again with the whole video (86 minutes). I waited for a while but it ended up getting all of the scenes right!
|
OK. I'll test with shorter clips and see what is happening there.
I've already realised that it will mess up badly if two short clips refer to same video media. That could cause more than one instance of ffprobe to try to write to same file which is nonsense.
Will update later today with version that works for shortened clips and shows some kind of progress indicator.
|
|
|
05-13-2019, 07:47 AM
|
#34
|
Human being with feelings
Join Date: Nov 2014
Location: Can Tho - Viet Nam
Posts: 305
|
I often forget to check the frame rate video, and cause unexpected errors. If you have the script detect frame rate video, then set the frame rate project to be compatible with the video on the track. I really need this
|
|
|
05-13-2019, 10:12 AM
|
#35
|
Human being with feelings
Join Date: Nov 2009
Location: UK
Posts: 669
|
Quote:
Originally Posted by swiiscompos
I'm on Mac. Ffprobe was still running in the Activity Monitor after I killed the process. Now the good news, I started the script again with the whole video (86 minutes). I waited for a while but it ended up getting all of the scenes right!
|
Just updated reapack. So refresh to get new version.
1. Now accounts properly for video offsets. So can handle cut up videos.
2. Only calls ffprobe once for each video source file.
3. Has (ugly) progress indicator (uses debug console).
It still processes the whole video file but there is a good reason for that.
That is so the cut numbering is consistent even for chopped up videos.
|
|
|
05-13-2019, 02:18 PM
|
#36
|
Human being with feelings
Join Date: Jan 2012
Location: Germany
Posts: 1,130
|
Superb script, mrlimbic!
Thanks a lot for sharing it.
I have a suggestion regarding the processing. Maybe you could add a little warning before the processing starts. Something like shitty Windows installers: "This might take a while, go and prepare some coffee." Also so that people know they can't work on other things in REAPER meanwhile.
|
|
|
05-13-2019, 04:57 PM
|
#37
|
Administrator
Join Date: Jan 2005
Location: NYC
Posts: 15,721
|
Hmm in theory one could write a video processor that does the detection (during playback), and sets some shared memory with the cut information that a ReaScript picks up and adds items or markers or splits...
Last edited by Justin; 05-13-2019 at 05:39 PM.
|
|
|
05-13-2019, 07:42 PM
|
#38
|
Human being with feelings
Join Date: Sep 2018
Location: China
Posts: 565
|
Quote:
Originally Posted by Justin
Hmm in theory one could write a video processor that does the detection (during playback), and sets some shared memory with the cut information that a ReaScript picks up and adds items or markers or splits...
|
Really? Video processor can do this? Which api is able to detect it?
|
|
|
05-14-2019, 12:55 AM
|
#39
|
Human being with feelings
Join Date: Apr 2013
Location: France
Posts: 9,875
|
@dysrock
You will have to write your own functions, based on pixel difference from one frame to another... That why it just 'in theory'.
But there is no real advantage of having this in video processor : you will be limited by playback speed, while ffprobe (used in this script) can do way faster analysis.
|
|
|
05-14-2019, 05:02 AM
|
#40
|
Human being with feelings
Join Date: Sep 2018
Location: China
Posts: 565
|
I have an idea of the script. I think it should detect the selected video items rather than all video items in the track.
|
|
|
Thread Tools |
|
Display Modes |
Linear Mode
|
Posting Rules
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is Off
|
|
|
All times are GMT -7. The time now is 07:20 AM.
|