I believe that there is a default (ie NEW) project template path in preferences -> Project
If you put together a template you want for all new projects you will have to save it as 'the' default project template, regardless of what you name it as, in the path designated for it in preferences - you can actually put it where you prefer on your pc as long the path is in the preferences project default.
Then, in Preferences -> General, Startup settings, open project on startup, choose what you want in the drop down menu. For you I guess that would be 'NEW'. Personally I always go for 'Last active project' then go for New Project in the File menu.
I assume that as you progress with a project you save it with a project name, not the project default template name.
I use a portable version of reaper so do not have the option in the start menu (windows?) although I know there are ways to work around this.
Excellent vid:
https://www.reaper.fm/videos.php#pMEcHXOsi1w that explains far better than me.Starts at about 4:17 but very worth watching the whole thing.
Sorry if I am going over stuff you already know but...