how to create a video clip using chroma-key

[mla]

In my previous post I asked for help with using chroma-key and provided two examples: one with default Reaper effects and a custom effect.

And the other with the same track order and effects that not works.

As there are no answers I thought that my description of the situation is not clear.

So, let me ask differently...

How to create a video using chroma-key in Reaper allowing to use any background?

As a start I did a video using a green screen background.

I would like to have step by step instructions allowing to easy manipulate with a cropping of the source video for smooth integration with a desired video or image background.

May be this topic is clearly described in Reaper manual?
I am ready to study the topic and experiment in order to aquire some basic and reliable skills.

What others do for using green screen?

Thx.

[mla]

here are two files the source image (object on green screen) and desired background image.

https://drive.google.com/drive/folde...iM?usp=sharing

I can apply chroma key, but green screen surrounding appears.

May be somebody can try to make it work and provide how to...

[papagirafe]

Originally Posted by mla

here are two files the source image (object on green screen) and desired background image.

https://drive.google.com/drive/folde...iM?usp=sharing

I can apply chroma key, but green screen surrounding appears.

May be somebody can try to make it work and provide how to...

Stock chromakey is hard to use and confusing. Just replace it with this version that allows you to mask out the unwanted parts as the first fx in your chain then add "overlay image" at the end of the chain and that should solve problem. Same project structure but you can eliminate the pure green track completely.

//Chroma-key(RGB) + mask out
//by papagirafe
//to be used as source for "overlay image" preset
//@param cl "mask left" 0 0 1 0.5 0.001
//@param cr "mask right" 0 0 1 0.5 0.001
//@param ct "mask top" 0 0 1 0.5 0.001
//@param cb "mask bottom" 0 0 1 0.5 0.001
//@param mode '0=green,1=blue' 0 0 1 0.5 1
//@param p primary 5.7 0 6 3 .01
//@param rej rejection 3 0 6 3 .01
//@param k3 threshold 0 -2 2 0 .01
//@param spill 'spill reduction' 0 0 1 0.5 1

in1=0;
in2=input_track(0);
colorspace='RGBA';
E=0xFFFE;
input_info(0,w,h)?(
  cl=(cl*w)&E;
  cr=(cr*w)&E;
  ct=(ct*h)&E;
  cb=(cb*h)&E;
  cx=cl;
  cy=ct;
  cw=w-(cl+cr);
  ch=h-(ct+cb); 
  gfx_set(0,mode?0:1,mode?1:0,1,0,-1,0);  //pure green/blue
  gfx_fillrect(0,0,w,h);    //fill all
  mode>.5?(k1=-rej;k2=p; ):(k1=p; k2=-rej;);
  gfx_keyedblit(0,cx,cy,cw,ch,cx,cy,k1,k2,k3,spill); //keyblit the cropped source over
);

[mla]

Originally Posted by papagirafe

Stock chromakey is hard to use and confusing. Just replace it with this version that allows you to mask out the unwanted parts as the first fx in your chain then add "overlay image" at the end of the chain and that should solve problem. Same project structure but you can eliminate the pure green track completely.

//Chroma-key(RGB) + mask out
//by papagirafe
//@param cl "mask left" 0 0 1 0.5 0.001
//@param cr "mask right" 0 0 1 0.5 0.001
//@param ct "mask top" 0 0 1 0.5 0.001
//@param cb "mask bottom" 0 0 1 0.5 0.001
//@param mode '0=green,1=blue' 0 0 1 0.5 1
//@param p primary 5.7 0 6 3 .01
//@param rej rejection 3 0 6 3 .01
//@param k3 threshold 0 -2 2 0 .01
//@param spill 'spill reduction' 0 0 1 0.5 1

in1=0;
in2=input_track(0);
colorspace='RGBA';
E=0xFFFE;
input_info(0,w,h)?(
  cl=(cl*w)&E;
  cr=(cr*w)&E;
  ct=(ct*h)&E;
  cb=(cb*h)&E;
  cx=cl;
  cy=ct;
  cw=w-(cl+cr);
  ch=h-(ct+cb); 
  gfx_set(0,1,0,1,0,-1,0);
  gfx_fillrect(0,0,w,h);
  mode>.5?(k1=-rej;k2=p; ):(k1=p; k2=-rej;);
  gfx_keyedblit(0,cx,cy,cw,ch,cx,cy,k1,k2,k3,spill);
);

Hi, thank you very much! Glad you are here , as last time you helped 100%.
I saved all the tips and screenshots from last troubleshooting.
Was sure it will work. But no way ...
What changed? My new green screen setup (different size and distance to it). So this is the main factor of the problem I guess.

Now... I did save your new code as preset, applied 3 presets in the order as previously, EXCLUDED green only track (that you mention).
Nothing works (see pics). I tried to play with all 3 presets and cannot even bring the saxman into the picture.

after trying different cropper software I found one that allows to do easy cropping.
Cropped video of saxman is on Green screen but the sides from surrounding are replaced by black. I think this cropping is not really helpful, so other techniques are required
Anyway, I cannot do anything with current presets.

I replace video track of saxman with a pic only.
I also attached project with new preset and background image.
Could you please apply your force/knowledge for trying to fix the situation.
What is your conclusion, would this initial green screen setup allow to do something useful?
Thanks again...

https://drive.google.com/drive/folde...usp=share_link

[papagirafe]

Originally Posted by mla

<snip>
I replace video track of saxman with a pic only.
I also attached project with new preset and background image.
Could you please apply your force/knowledge for trying to fix the situation.
What is your conclusion, would this initial green screen setup allow to do something useful?
Thanks again...

https://drive.google.com/drive/folde...usp=share_link

Try this version of the project (btw I set the project's to 1920x1080):
https://stash.reaper.fm/46119/fixed%20green.rpp

That should also correct the sizing problems. :-)

[mla]

I think you forgot to attach the file...

should be changed saxman pic. when open your rpp it shows

[mla]

My appologies...
When complained in previous message I provided the path to my file
and top and the bottom of saxman were cut out...
So I panicked (thought that you adjusted the pic prior to provide new rpp).
Than tried your Croma-key handles: mask left righ top and bottom
Works like a charm...

I will try to recreate a project from scratch following your example.

Tell me please... should I crop my green screen video and leave saxman on green with black surrounding that basically replaced room view.
Or using suggested effects it doesn't matter and it should simply work?

Thank you very much. Sure you deserve a good beer .
Promised! Let me just practice and confirm that I can manage this green screen setup without any issue.

[papagirafe]

Originally Posted by mla

My appologies...
When complained in previous message I provided the path to my file
and top and the bottom of saxman were cut out...
So I panicked (thought that you adjusted the pic prior to provide new rpp).
Than tried your Croma-key handles: mask left righ top and bottom
Works like a charm...

I will try to recreate a project from scratch following your example.

Tell me please... should I crop my green screen video and leave saxman on green with black surrounding that basically replaced room view.
Or using suggested effects it doesn't matter and it should simply work?

Thank you very much. Sure you deserve a good beer .
Promised! Let me just practice and confirm that I can manage this green screen setup without any issue.

Glad to be of help! I updated the code to remove some inconsistencies. The preset masks out the unwanted sections with pure green/blue and then chroma blits the relevent section of your source. No need for background image at all.
As for the beer... next time I'm in Toronto for sure! :-)

[mla]

Originally Posted by papagirafe

Glad to be of help! I updated the code to remove some inconsistencies. The preset masks out the unwanted sections with pure green/blue and then chroma blits the relevent section of your source. No need for background image at all.
As for the beer... next time I'm in Toronto for sure! :-)

I tested few vids and backgrounds quickly.
You did absolutely great job.
I have some colors instability with lighter background that I could not make flawless. Will try improve green screen approach.
I have very basic knowledge in that. I had just 20cm distance between saxman and green screen. This I can improve.
Will experiment a bit.
But the code you wrote and whole approach seems to be absolutely smart and pro.
I am in Montreal not in Toronto . So check please your PayPal from Michael A. You have your beer or good Espresso with a piece of cake (your choice).
Thank you very much for friendly help!

[papagirafe]

Originally Posted by mla

I tested few vids and backgrounds quickly.
You did absolutely great job.
I have some colors instability with lighter background that I could not make flawless. Will try improve green screen approach.
I have very basic knowledge in that. I had just 20cm distance between saxman and green screen. This I can improve.
Will experiment a bit.
But the code you wrote and whole approach seems to be absolutely smart and pro.
I am in Montreal not in Toronto . So check please your PayPal from Michael A. You have your beer or good Espresso with a piece of cake (your choice).
Thank you very much for friendly help!

Thanx for the espresso! Montreal? I go there more often than Toronto!

[mla]

Originally Posted by papagirafe

Thanx for the espresso! Montreal? I go there more often than Toronto!

I will send you PM

[mla]

Originally Posted by papagirafe

Glad to be of help! I updated the code to remove some inconsistencies. The preset masks out the unwanted sections with pure green/blue and then chroma blits the relevent section of your source. No need for background image at all.
As for the beer... next time I'm in Toronto for sure! :-)

Hi,
I did few tests and played with wonderful presets that you combined.
Masks and zoom performs 100%.
So, my big problem gone. But now I have another one I cannot tweak in any way my shoes part . "Primary" dialer in chroma preset not allows me to find nice looking shoes.

I did couple of vids. First blamed the light. Now I have about no shadows on the screen. I think my first photo on green screen which you used for fixing was in no way better. But there was no problem with as now. The only difference is that the shoes were white-grey instead of black now. Could it be the culprit? I attach the proj with short vid. Check please the vid itself in the folder prior to run it in the project. Is my green screen setup is really bad and this cause the shoes problem?
May be you can suggest something that will reliably help to fix this kind of things?
Thx.


https://drive.google.com/drive/folde...hB?usp=sharing

[papagirafe]

Originally Posted by mla

Hi,
I did few tests and played with wonderful presets that you combined.
Masks and zoom performs 100%.
So, my big problem gone. But now I have another one I cannot tweak in any way my shoes part . "Primary" dialer in chroma preset not allows me to find nice looking shoes.

I did couple of vids. First blamed the light. Now I have about no shadows on the screen. I think my first photo on green screen which you used for fixing was in no way better. But there was no problem with as now. The only difference is that the shoes were white-grey instead of black now. Could it be the culprit? I attach the proj with short vid. Check please the vid itself in the folder prior to run it in the project. Is my green screen setup is really bad and this cause the shoes problem?
May be you can suggest something that will reliably help to fix this kind of things?
Thx.


https://drive.google.com/drive/folde...hB?usp=sharing

The RGB version of this filter seems to have limits but the YV12/YUY2 mode seems more adjustable. Unfortunately I would need to use new magic to make my code work...

[mla]

Originally Posted by papagirafe

The RGB version of this filter seems to have limits but the YV12/YUY2 mode seems more adjustable. Unfortunately I would need to use new magic to make my code work...

So I have a chance to get it?
Second Espresso promised

[papagirafe]

Originally Posted by mla

So I have a chance to get it?
Second Espresso promised

I looked at the implication of this mode: the code structure needs to be changed substancially to include a beefed up version of "overlay image" inside. Very feasible but not a 15 minutes job :-)

[mla]

I see...
I just thought that if my initial green screen is not so bad from basic techical requirements perspective the appropriate program should easily do the trick. A friend of mine recomended DAVINCHI.
The installation file is 3.2GB... it's funny. Then a learning courve, that I am shure will not be so simple as in Reaper (of course with top notch presets if avalable).
I guess for my needs should be a simpler solution.
There are VideoPad and Videoproc apps with tube tutorials.

Ideally, I would like to use Reaper because I love it for audio and 6 month ago you helped with my first video thing. May be I had better green screen vid for start...
Anyway, if by chance you'll have time to try to organise presets that will flawlessly manage all this "green" stuff it would be nice...

I really would like to be able to make this in Reaper.
For now, will continue to watch this thread in hope of final solution.
Thank you for your time and explanation.

Best!

[mla]

Hi,
I digged in a bit more with VIDEOPROC...
Absolutely simple and good quality for my green screen needs.
Few clicks and export...
I have to play with audio made in Reaper but may be will drop ready video to Reaper with audio proj.


There are other tweaking effects but even basic clicks give a very good result IMHO.
As for Reaper, I think that if you will succeed to adjust more controlable tweaking of Green it will be great.
Sure I would try new shiny presets if you'll do some...

Here is a screenshot from the vid. No probs with green at all.
https://drive.google.com/drive/folde...IP?usp=sharing

Thanks.

[mla]

here is my "creative" vid with chroma applied.

https://youtu.be/Z-OSwmEWWaQ

I would appreciate any comment/suggestion related to green.

Sure, you don't have to listen my music .

[papagirafe]

Originally Posted by mla

<snip>
Ideally, I would like to use Reaper because I love it for audio and 6 month ago you helped with my first video thing. May be I had better green screen vid for start...
Anyway, if by chance you'll have time to try to organise presets that will flawlessly manage all this "green" stuff it would be nice...

I really would like to be able to make this in Reaper.
For now, will continue to watch this thread in hope of final solution.
Thank you for your time and explanation.

Best!

I was finally able to squeeze some coding time in my schedule (the summer vacations just finished here... yeah it's reversed seasons in Brasil). Here is the YUV version that acts as a "overlay image" on steroids with a Chroma-key twist to it. It takes care of size mismatch between project_w/h, foreground and background. The special parameter "sizing policy" will set the base zoom to match your needs (0=no change,1=maximize image to project height,2=max to project width,3=stretch). I feel this is what the original preset should have been. This preset works as a standalone and it should give better results. BTW Don't forget to set your project's dimensions or it will default to 1280x720.

//Overlay: image w/YUV chroma-key
//by papagirafe
//if you like this preset, please consider a donation or a tip at https://paypal.me/papagirafe

//@param hue 'hue: g=-136,b=0' -136 -180 180 0 1
//@param mag saturation .7 0 1.0 0.5 .01
//@param thresh threshold .41 0 1 .5 .01
//@param gain gain 1.8 0 8 4 .1

//@param6:fg.sp "fg sizing policy" 0 0 3 1.5 1    //0:as is, 1:maximize height, 2:max width, 3:fit project
//@param fg.z "fg zoom" 0 -4 4 0 0.001
//@param fg.xo "fg x offset %" 0 -100 100 0 0.1
//@param fg.yo "fg y offset %" 0 -100 100 0 0.1
//@param fg.ml "mask left %" 0 0 50 25 0.1
//@param fg.mr "mask right %" 0 0 50 250.1
//@param fg.mt "mask top %" 0 0 50 25 0.1
//@param fg.mb "mask bottom %" 0 0 50 25 0.1

//@param15:bg "bg input track" 1 1 20 10 1 
//@param bg.sp "bg sizing policy" 0 0 3 1.5 1     //0:as is, 1:maximize height, 2:max width, 3:fit project
//@param bg.z "bg zoom" 0 -2 2 0 0.001
//@param bg.xo "bg x offset %" 0 -100 100 0 0.1
//@param bg.yo "bg y offset %" 0 -100 100 0 0.1

//pre-scaling
fg.xo/=100;fg.yo/=100;fg.ml/=100;fg.mr/=100;fg.mt/=100;fg.mb/=100;bg.xo/=100;bg.yo/=100;

function gfx_superblit(keyed)
  instance(sp,z,w,h,xo,yo,ml,mr,mt,mb)
  local(hz,vz,dx,dy,dw,dh,mx,my,mw,mh,i)
  global(project_w,project_h,gfx_dest,hue,mag,thresh,gain)
(
  sp==0?(hz=vz=10^z):
  sp==1?(hz=vz=(project_h/h)*10^z):
  sp==2?(hz=vz=(project_w/w)*10^z):
  (hz=(project_w/w)*10^z; vz=(project_h/h)*10^z);
  mx=w*ml; 
  my=h*mt;
  mw=max(0,w-(w*ml+w*mr));
  mh=max(0,h-(h*mt+h*mb));
  dw=hz*mw; dh=vz*mh;
  dx=(project_w-dw)/2 + project_w*(xo) + mx*hz/2 - mr*w*hz/2; 
  dy=(project_h-dh)/2 + project_h*(yo) + my*vz/2 - mb*h*vz/2;

  keyed?(
    //due to keyedblit() parameter limitation, we need a temporary image
    gfx_dest=i=gfx_img_alloc(dw,dh); 
    gfx_blit(this,0, 0,0,dw,dh, mx,my,mw,mh); //controlled resize blit
    gfx_dest=-1; 
    gfx_keyedblit(i, dx,dy,dw,dh, 0,0, cos(hue*$pi/180)*mag,sin(hue*$pi/180)*mag,thresh,gain);
    gfx_img_free(i);
  ):gfx_blit(this,0, dx,dy,dw,dh, mx,my,mw,mh); //controlled resize blit
);

!project_wh_valid?(project_w=1280;project_h=720);
input_info(fg=0,fg.w,fg.h)?(
  colorspace='YV12';
  gfx_fillrect(0,0,project_w,project_h);
  input_info(bg,bg.w,bg.h)?bg.gfx_superblit(0);
  fg.gfx_superblit(1);
);

__________________________________________________ ________________________________________
if you find this module helpful please consider a contribution, even small, to help support further cool developments! :-)

[mla]

just checked the forum...
I will try the new code and let you know.
Would be fun to have it equal to the app I used.

Thank you very much for your time.

[papagirafe]

Originally Posted by mla

just checked the forum...
I will try the new code and let you know.
Would be fun to have it equal to the app I used.

Thank you very much for your time.

The YUV mode has a lot more precision on color & saturation. You should be able to obtain the same results. Can´t wait to know if it works!

[mla]

Hi,
I spent some time trying... cannot make it work.
I am messing with the same presets on vid and background tracks, just replaced your previous with a new one.
I get some weird result...
Then I tried only your new one. The same.

May be you can to do it and upload the proj, so I can check FXs on each track.

In case you want to try... I uploaded exact same background image and 15 sec of the video I used in another program were I got perfect result green screen wise...
https://drive.google.com/drive/folde...usp=share_link

[papagirafe]

Originally Posted by mla

Hi,
I spent some time trying... cannot make it work.
I am messing with the same presets on vid and background tracks, just replaced your previous with a new one.
I get some weird result...
Then I tried only your new one. The same.

May be you can to do it and upload the proj, so I can check FXs on each track.

In case you want to try... I uploaded exact same background image and 15 sec of the video I used in another program were I got perfect result green screen wise...
https://drive.google.com/drive/folde...usp=share_link

I just tried with almost immediate success. Here are the sweet parameters. (Beware of the project´s setting "Video item visivbility"). You may open the image in a sperate browser tab to view in full resolution. (It has been tested at the item level, not track)

[mla]

see the screenshot...
I copied your FX application... see "no me" file

https://drive.google.com/drive/folde...IP?usp=sharing

Green screen content is not showing up at all...
What I am missing here?

[papagirafe]

Originally Posted by mla

see the screenshot...
I copied your FX application... see "no me" file

https://drive.google.com/drive/folde...IP?usp=sharing

Green screen content is not showing up at all...
What I am missing here?

Here is the project I made for testing:
https://stash.reaper.fm/46232/green3.rpp
check if you have the same project settings. If you are using the default item priority, track order have to be reversed.

[mla]

Originally Posted by papagirafe

Here is the project I made for testing:
https://stash.reaper.fm/46232/green3.rpp
check if you have the same project settings. If you are using the default item priority, track order have to be reversed.

I am pretty sure I use default item priority
Will check a proj...

Could you provide track order for default priority ...
Thank you!

[mla]

compared project settings yours against mine...
they are exactly the same beside on media tab I selected 24 wav...
sure it has no impact...
What parameter in Project settings I have to look for or may be the other place in the the project are configured differently in yours against default (or affecting tracks that you mentioned)

[papagirafe]

Originally Posted by mla

compared project settings yours against mine...
they are exactly the same beside on media tab I selected 24 wav...
sure it has no impact...
What parameter in Project settings I have to look for or may be the other place in the the project are configured differently in yours against default (or affecting tracks that you mentioned)

Could you send me (even though a private message, yes it is possible) a link to your rpp file

[mla]

https://drive.google.com/drive/folde...usp=share_link

[papagirafe]

Originally Posted by mla

compared project settings yours against mine...
they are exactly the same beside on media tab I selected 24 wav...
sure it has no impact...
What parameter in Project settings I have to look for or may be the other place in the the project are configured differently in yours against default (or affecting tracks that you mentioned)

You did miss one critical parameter ;-). Here is a graphical explanation :-)



more detailed view

[mla]

Thank you very much for troubleshooting

I will try later to play with dials.

[papagirafe]

Originally Posted by mla

Thank you very much for troubleshooting

I will try later to play with dials.

You are not the only one to be confused by this setting, worth the extra-mile for future reference :-)

[mla]

I confirm that it works flawlessly.

I don't think that I could lose the quality during effect application compare to Videoproc.
One thing that I noticed that after importing final video from Videoproc to Reaper (I did it for synching audio, did't find a way to do it so presisely as in Reaper) and then rendering imported video with adjusted audio, the size of the file became about twice bigger and it's with the same resoluton.
I have to double check that.

Even it's very simple with fancy graphic interface (sure, it's video editor) I would use Reaper after few experiments.
If needed I can import final Reaper file and import it to Videoproc for conversion with smaller file size.
Anyway THANK YOU very much.
Sure you know the thing from A to Z and it is pretty nimble solution that many can use.
I send something for the second esperesso with a piece of cake.

[mla]

Originally Posted by papagirafe

You are not the only one to be confused by this setting, worth the extra-mile for future reference :-)

BTW, the FX is applied on item in your example.
Would it work the same way if applied on track?

If no probs - OK, if it's required to be on item, please explain why.

Just cannot try now, but interested to know.

[papagirafe]

Originally Posted by mla

I confirm that it works flawlessly.

I don't think that I could lose the quality during effect application compare to Videoproc.
One thing that I noticed that after importing final video from Videoproc to Reaper (I did it for synching audio, did't find a way to do it so presisely as in Reaper) and then rendering imported video with adjusted audio, the size of the file became about twice bigger and it's with the same resoluton.
I have to double check that.

Even it's very simple with fancy graphic interface (sure, it's video editor) I would use Reaper after few experiments.
If needed I can import final Reaper file and import it to Videoproc for conversion with smaller file size.
Anyway THANK YOU very much.
Sure you know the thing from A to Z and it is pretty nimble solution that many can use.
I send something for the second esperesso with a piece of cake.

Thank for the tip! As for file size if you upgrade the ffmpeg module you get small file size ang good quality with the h.264 encoder. The upgrade process has been described on video at the (unofficial) Reaper blog

[papagirafe]

Originally Posted by mla

BTW, the FX is applied on item in your example.
Would it work the same way if applied on track?

If no probs - OK, if it's required to be on item, please explain why.

Just cannot try now, but interested to know.

Most probably yes but untested. As a general rule FX on items are way better for video unless you have a strong reason not to do so. Video and audio do not handle "emptyness" the same way and often track video fx create unwanted side effects.

[mla]

Originally Posted by papagirafe

Most probably yes but untested. As a general rule FX on items are way better for video unless you have a strong reason not to do so. Video and audio do not handle "emptyness" the same way and often track video fx create unwanted side effects.

Thank you very much for the explanation... sure, after it I don't have to try any testing

[mla]

Originally Posted by papagirafe

As for file size if you upgrade the ffmpeg module you get small file size ang good quality with the h.264 encoder. The upgrade process has been described on video at the (unofficial) Reaper blog

could you provide a link for the best tutorial plz?

[mla]

Originally Posted by mla

could you provide a link for the best tutorial plz?

I found this one... https://reaper.blog/2021/12/ffmpeg-for-reaper-643/

will try tomorrow

Thank you for all the advices!

[papagirafe]

Originally Posted by mla

I found this one... https://reaper.blog/2021/12/ffmpeg-for-reaper-643/

will try tomorrow

Thank you for all the advices!

Yes the second one on this page is the procedure to install. Makes a *big* difference in performance and quality.