Stream video and stereo audio via browser

svijayrathinam · 03-27-2018, 01:49 PM

Hi,

I would love to see a feature where I can stream my video and a stereo master in a project with a plugin.. Which should generate a link where you can see the broadcast from a browser. This will be so useful for client remote reviews. I can simply put the plugin on the master track and stream video and audio and my client can review my work via a browser...

SonicAxiom · 03-27-2018, 08:00 PM

++++++++ absolutely great idea!!!!

I'm currently using old Windows Media Encoder to stream Reaper's master output (or other audio signals) over the internet as an mp3 audio stream (in fact wma). WME offers to choose a file or a hardware audio/vieo input as the source for streaming.

There should be an option in Reaper to stream audio from any desired point in the signal flow (not only master output) and to optionally encode this stream to a user configurable mp3/opus stream on the fly (favorably multichannel-capable also).

WME does real peer-to-peer streaming without any third-party server in between. I simply provide my public IP address (in fact a dynamic dns human-readable link address) and the client can directly connect to my stream. The streaming quality is very good! A streaming solution in Reaper should absolutely offer an option to stream without the need for a third-party server like icecast/shoutcast.

I was just about to file a feature request to be able to use dns names instead of ip addresses in ReaStream and to add optional, configurable mp3 compression functionality to it. I managed to connect to a remote friend via ReaStream but the uncompressed (and maybe unbuffered) stereo stream is jamming my internet connection in spite of my 1.2 MB/sec. upstream capacity.

.

echoblaster · 08-13-2018, 01:21 AM

Fantastic idea. Did you ever make a feature request? This so much needs to be a part of Reaper. The functionality is almost there.

svijayrathinam · 08-13-2018, 01:22 AM

Quote:

Originally Posted by echoblaster

Fantastic idea. Did you ever make a feature request? This so much needs to be a part of Reaper. The functionality is almost there.

What do you mean the functionality is almost there ? Can you pls explain ?

By the way..this post is on feature request section of the forum

Meo-Ada Mespotine · 08-13-2018, 04:47 AM

+1 on this. Audio is already possible with the shoutcast-plugin, but video should be doable as well.

As far as I know, there is also a way to access the video-chain in Reaper from VST-Plugins, so it could be doable by us.
Though this is totally underdocumented...

svijayrathinam · 08-13-2018, 04:50 AM

Quote:

Originally Posted by mespotine

+1 on this. Audio is already possible with the shoutcast-plugin, but video should be doable as well.

As far as I know, there is also a way to access the video-chain in Reaper from VST-Plugins, so it could be doable by us.
Though this is totally underdocumented...

Some info on this would help a great deal...

Meo-Ada Mespotine · 08-13-2018, 04:54 AM

All I found was two files(probably on Justin's dev-server, but I'm not sure) on that 'til now.
The comments hint at the possibility in vst-plugins:

video_frame.h

Code:

#ifndef _REAPER_VIDEO_FRAME_H_
#define _REAPER_VIDEO_FRAME_H_

class IVideoFrame 
{
  public:

  double time_start WDL_FIXALIGN;
  double time_end;

    virtual void AddRef()=0;
    virtual void Release()=0;
    virtual char *get_bits()=0;
    virtual int get_w()=0;
    virtual int get_h()=0;
    virtual int get_fmt()=0; // 'YV12', 'YUY2', 'RGBA' are valid
    virtual int get_rowspan()=0;
    virtual void resize_img(int wantw, int wanth, int wantfmt)=0;

    virtual INT_PTR Extended(int code, INT_PTR parm1, INT_PTR parm2, void* parm3) { return 0; }
};


#endif

video_processor.h

Code:

#ifndef _REAPER_VIDEO_PROCESSOR_H_
#define _REAPER_VIDEO_PROCESSOR_H_

#include "video_frame.h"

class IREAPERVideoProcessor
{
public:
  IREAPERVideoProcessor() { process_frame=NULL; get_parameter_value = NULL; userdata = NULL; }
  virtual ~IREAPERVideoProcessor() { }

  enum { REAPER_VIDEO_PROCESSOR_VERSION=0x100 }; 

  // give callbacks to query incoming video frames
  // + ownedSourceInst API + render

  virtual IVideoFrame *newVideoFrame(int w, int h, int fmt)=0; // always use this for IVideoFrames returned by process_frame, never create your own. return it from process_frame(), or Release() it.

  virtual int getNumInputs()=0;
  virtual int getInputInfo(int idx, void **itemptr)=0; // input can be -1 for current context. sets itemptr if non-NULL. trackidx can be 100000000 for preview, or other values
  virtual IVideoFrame *renderInputVideoFrame(int idx, int want_fmt/*0 for native*/)=0; // must treat returned frame as immutable! either return it or Release() it.

  // user-specified
  IVideoFrame *(*process_frame)(IREAPERVideoProcessor *vproc, 
                                const double *parmlist, int nparms,  // first parameter is wet/dry, then subsequent map to that of the plug-in  (up to VIDEO_EFFECT_MAX_PARAMETERS)
                                double project_time, double frate, 
                                int force_format // 0, or 'YV12', 'RGBA', 'YUY2'
                              ); // return video frame

  bool (*get_parameter_value)(IREAPERVideoProcessor *vproc, int idx, double *v); // queries the value of a parameter from the video thread. return false if parameter out of bounds

  void *userdata;

};

/*
 To use IREAPERVideoProcessor in a VST:


    void *ctx=(void*)hostcb(&m_effect,0xdeadbeef,0xdeadf00e,4,NULL,0.0f);
    if (ctx)
    {
      IREAPERVideoProcessor *(*video_CreateVideoProcessor)(void *fxctx, int version);
      *(void **)&video_CreateVideoProcessor = (void *)hostcb(&m_effect,0xdeadbeef,0xdeadf00d,0,"video_CreateVideoProcessor",0.0f);
      if (video_CreateVideoProcessor)
      {
        m_videoproc = video_CreateVideoProcessor(ctx,IREAPERVideoProcessor::REAPER_VIDEO_PROCESSOR_VERSION);
        if (m_videoproc)
        {
          m_videoproc->userdata = _this;
          m_videoproc->process_frame = staticProcessFrame;
          m_videoproc->get_parameter_value = staticGetVideoParam;
        }
      }
    }


  static IVideoFrame *staticProcessVideoFrame(IREAPERVideoProcessor *vproc, const double *parmlist, int nparms, double project_time, double frate,  int force_format)
  {
    IVideoFrame *vf = vproc->newVideoFrame(320,200,'RGBA');
    if (vf)
    {
    // bla blah. parmlist[0] is wet, [1] is parameter, values at video time (which may differ depending on audio). do not update internal state, but use those parameters for rendering
    }
    return vf;
  }

    static bool staticGetVideoParam(IREAPERVideoProcessor *vproc, int idx, double *valueOut)
    {
      // called from video thread, gets current state
      if (idx>=0 && idx < NUM_PARAMS)
      {
        *valueOut = m_parms[idx];
         return true;
      }
      return false;
    }

  */


#endif

analogexplosions · 08-13-2018, 11:21 AM

This would be absolutely incredible for me if it could work!

svijayrathinam · 08-14-2018, 01:16 PM

Quote:

Originally Posted by mespotine

All I found was two files(probably on Justin's dev-server, but I'm not sure) on that 'til now.
The comments hint at the possibility in vst-plugins:

video_frame.h

Code:

#ifndef _REAPER_VIDEO_FRAME_H_
#define _REAPER_VIDEO_FRAME_H_

class IVideoFrame 
{
  public:

  double time_start WDL_FIXALIGN;
  double time_end;

    virtual void AddRef()=0;
    virtual void Release()=0;
    virtual char *get_bits()=0;
    virtual int get_w()=0;
    virtual int get_h()=0;
    virtual int get_fmt()=0; // 'YV12', 'YUY2', 'RGBA' are valid
    virtual int get_rowspan()=0;
    virtual void resize_img(int wantw, int wanth, int wantfmt)=0;

    virtual INT_PTR Extended(int code, INT_PTR parm1, INT_PTR parm2, void* parm3) { return 0; }
};


#endif

video_processor.h

Code:

#ifndef _REAPER_VIDEO_PROCESSOR_H_
#define _REAPER_VIDEO_PROCESSOR_H_

#include "video_frame.h"

class IREAPERVideoProcessor
{
public:
  IREAPERVideoProcessor() { process_frame=NULL; get_parameter_value = NULL; userdata = NULL; }
  virtual ~IREAPERVideoProcessor() { }

  enum { REAPER_VIDEO_PROCESSOR_VERSION=0x100 }; 

  // give callbacks to query incoming video frames
  // + ownedSourceInst API + render

  virtual IVideoFrame *newVideoFrame(int w, int h, int fmt)=0; // always use this for IVideoFrames returned by process_frame, never create your own. return it from process_frame(), or Release() it.

  virtual int getNumInputs()=0;
  virtual int getInputInfo(int idx, void **itemptr)=0; // input can be -1 for current context. sets itemptr if non-NULL. trackidx can be 100000000 for preview, or other values
  virtual IVideoFrame *renderInputVideoFrame(int idx, int want_fmt/*0 for native*/)=0; // must treat returned frame as immutable! either return it or Release() it.

  // user-specified
  IVideoFrame *(*process_frame)(IREAPERVideoProcessor *vproc, 
                                const double *parmlist, int nparms,  // first parameter is wet/dry, then subsequent map to that of the plug-in  (up to VIDEO_EFFECT_MAX_PARAMETERS)
                                double project_time, double frate, 
                                int force_format // 0, or 'YV12', 'RGBA', 'YUY2'
                              ); // return video frame

  bool (*get_parameter_value)(IREAPERVideoProcessor *vproc, int idx, double *v); // queries the value of a parameter from the video thread. return false if parameter out of bounds

  void *userdata;

};

/*
 To use IREAPERVideoProcessor in a VST:


    void *ctx=(void*)hostcb(&m_effect,0xdeadbeef,0xdeadf00e,4,NULL,0.0f);
    if (ctx)
    {
      IREAPERVideoProcessor *(*video_CreateVideoProcessor)(void *fxctx, int version);
      *(void **)&video_CreateVideoProcessor = (void *)hostcb(&m_effect,0xdeadbeef,0xdeadf00d,0,"video_CreateVideoProcessor",0.0f);
      if (video_CreateVideoProcessor)
      {
        m_videoproc = video_CreateVideoProcessor(ctx,IREAPERVideoProcessor::REAPER_VIDEO_PROCESSOR_VERSION);
        if (m_videoproc)
        {
          m_videoproc->userdata = _this;
          m_videoproc->process_frame = staticProcessFrame;
          m_videoproc->get_parameter_value = staticGetVideoParam;
        }
      }
    }


  static IVideoFrame *staticProcessVideoFrame(IREAPERVideoProcessor *vproc, const double *parmlist, int nparms, double project_time, double frate,  int force_format)
  {
    IVideoFrame *vf = vproc->newVideoFrame(320,200,'RGBA');
    if (vf)
    {
    // bla blah. parmlist[0] is wet, [1] is parameter, values at video time (which may differ depending on audio). do not update internal state, but use those parameters for rendering
    }
    return vf;
  }

    static bool staticGetVideoParam(IREAPERVideoProcessor *vproc, int idx, double *valueOut)
    {
      // called from video thread, gets current state
      if (idx>=0 && idx < NUM_PARAMS)
      {
        *valueOut = m_parms[idx];
         return true;
      }
      return false;
    }

  */


#endif

All this looks too daunting for me..Any scripters can help here pls ??

Meo-Ada Mespotine · 08-14-2018, 01:33 PM

Only C/C++-coders can do something with that.

Way beyond the scope of scripters (like me)

echoblaster · 08-15-2018, 01:38 PM

Quote:

Originally Posted by svijayrathinam

What do you mean the functionality is almost there ? Can you pls explain ?

By the way..this post is on feature request section of the forum

There's almost functionality with audio via Reastream (I've yet to get it to work though).

svijayrathinam · 08-20-2018, 01:59 PM

Quote:

Originally Posted by echoblaster

There's almost functionality with audio via Reastream (I've yet to get it to work though).

I havent figure out reastream either..But if reaper allows streaming of video and audio..Nothing like it !!

airon · 08-20-2018, 09:46 PM

OBS Studio might work better. It supports so many streaming platfoms, bitrates, has zillions of tutorials on Youtube, can capture Windows, desktops or areas, can use ASIO via the ASIO plugin, can even use VST plugins.....

It might be more practical just for its flexible video encoding setup.