aligned video-patch with visible edges

Q&A about the latest versions
Tawrank
Posts: 12
Joined: Fri Feb 15, 2019 9:28 pm

hi, i followed the tutorial about aligning video, quite complicated, but i succeeded so far. the only problem is, when i pin the video to the panorama, i can see the harsh edges of the patch, due to different colour and brightness. i mean, the patch runs through different processes in ptgui, photoshop and pano2vr ... is it the final rendering that leads to these changes in colour ? the edges of the final patch are part of the extracted patch from the original panorama, not from the video, but at the end, both, the patch and the image sequence, run through the final rendering, to get the video. can anybody give me an advice how the problem could be eliminated ?
User avatar
Hopki
Gnome
Posts: 13005
Joined: Thu Jan 10, 2008 3:16 pm
Location: Layer de la Haye, Essex UK
Contact:

Hi,
We are looking at an alternative way of doing this.
The biggest problem with this is the colour match for image and a video.
It is also down to each browser which will process the videos and pictures so you will see differences between browser.
So even though you get it perfect for say Edge, it will probably look bad in Firefox.
It does depend on the scene, on how bad it looks.

It's not just Pano2VR I have seen some really bad examples on all players/tour builders that can do this.
As and when we have something we will blog about it.
Regards,
Hopki
Garden Gnome Support
If you send an e-mail to support please send a link to the forum post for reference.
support@ggnome.com
https://ggnome.com/wiki/documentation/
User avatar
TheRealMojoJojo
Posts: 341
Joined: Thu Sep 20, 2018 10:18 pm

Yeah, the best option I have found is minimal editing of the original pan, however, your patch image itself should always match up perfectly with the panorama with no color distortion other than maybe some of video that's showing through the patch.....but the patch edges should always match up to the panorama.
patch-fishtank.jpg
patch-fishtank.jpg (494.9 KiB) Viewed 5750 times
Shot with a Pentax-K1 + HD8M-P 8mm f/3.5 HD Fisheye Lens

In that screenshot all of the tanks are showing through the patch leaving the edges of the patch to match up perfectly. Be sure that when you are editing the video/patch layer that you do not remove any of the patch layers edges.

It's tough to get the video to match if you've altered the original pan by running filters in photoshop or whatever you use for your image editing and forget about it if you shot the pan in low lighting taking bracketed images with a wide angle lens because the video will never contain the same amount of viable information as the raw images which makes it impossible to match up because the video is going to be grainy while the panorama is not.

Also, I've found it useful to use Photoshop's "curves" on the video layer image frame to match the image sequence with the patch the best I can before rendering the image sequence to a video file. You will first have to convert the image sequence to a "smart object". After you have converted it to a smart object the edits that you make to one frame with carry over to all the rest of the frames when you render the project.

There are a lot of variables but like I said in the beginning, minimal image editing for the panorama otherwise you will throw the color, contrast, white balance and everything else off and you won't be able to efficiently do the same to the video file because it just doesn't contain the same amount of info per frame.
Tawrank
Posts: 12
Joined: Fri Feb 15, 2019 9:28 pm

@ Hopki: thanks for your reply, that is not satisfying but good to know. I could not explain that to my client, who would f.e. welcome the tour-visitors in the first node. looking forward to you finding a solution ...

@ 1uv3: hi there, as i explained it´s the patch edges, that do not match, although they should, as you mentioned it. that´s why i supposed, the colour distortion of the patch-edges happens in the final rendering. the differences between patch and video are evident to me, nevertheless thanks for your advices concerning that issue. by the way: when i do the first step, to create the image sequence, the imported video-layers (taken with a canon 5d mark3) look terrible in colour, without any profile. here i check around with the different coulour spaces to get already closer to the extracted patch, which i use later on. I will have another try today, but as Hopki said, up to now the problem can´t be fixed, for reason of the display in different browsers ...
User avatar
TheRealMojoJojo
Posts: 341
Joined: Thu Sep 20, 2018 10:18 pm

Hey there,

Maybe I am misunderstanding what you are talking about but if the edges of your patch aren't color matching the pano then something is going wrong in your process. I've tested across all browser platforms and have the same results with no detection that the patch is even there aside from the video playing and definitely cannot see where the pano ends and the patch begins.

Here is a short video to show you what I mean https://youtu.be/j6AZyEvWUzk

I didn't think about checking on the other browsers till afterwards but here are the screenshots:

Opera
opera.jpg
opera.jpg (442.65 KiB) Viewed 5732 times
FF
firefix.jpg
firefix.jpg (428.03 KiB) Viewed 5732 times
Safari
safari.jpg
safari.jpg (408.91 KiB) Viewed 5732 times
Tawrank
Posts: 12
Joined: Fri Feb 15, 2019 9:28 pm

hey 1uv3,
no, you aren´t misunderstanding, i really have the problem, that the edges of the patch are visible, not matching with the pano. i checked to pin the extracted patch from ptgui directly to the pano, without merging with the video and everything was fine. after rendering the imagesequence together with the patch to create the final video, the problem exists. would you please tell me, which rendering-options you use. thank you for your examples, i am really wondering why your edges look that good, even in different browsers. when i compare the examples of the blacksmith on the gnome-site, in firefox and safari, i even see big differences ... i have no idea anymore, where my mistake lies, would be great, if you can help me to figure this out.
Tawrank
Posts: 12
Joined: Fri Feb 15, 2019 9:28 pm

Hi once again,
just watched your video - no problem to follow every step, thank you for that ! i guess the problem is my Photoshop version, which is CS5 and has not the same possibilities like yours. F.e. I can´t choose the Adobe Media Encoder nor can i set a hook to Colour Manage. What i can do, is exporting the video as a QuickTime-movie - this must be the reason !
User avatar
TheRealMojoJojo
Posts: 341
Joined: Thu Sep 20, 2018 10:18 pm

Hi,

Quicktime could definitely be the culprit. I am going to pm you.

I found that only Photoshop CC 2017 works for the live pano process. I am using Windows version. For some reason, other Photoshops wouldn't even allow me to import videos for image sequencing and are also missing the check box that enables you to import the image sequence towards the final steps.
Tawrank
Posts: 12
Joined: Fri Feb 15, 2019 9:28 pm

Hi, i´m still working on it, so other questions are coming up, during the process ... I imported a video to photoshop in the first step and got an image-file with 268 layers, which means 10 seconds plus 18 frames of film, filmed with 25 fps. When i then exported these layers as an image-sequence i got only 223 images, means 45 images are lost. This must also be the reason, why f.e. spoken text is not synchron with movement of the mouth ... Did this happen to anyone else ? Greets !
User avatar
TheRealMojoJojo
Posts: 341
Joined: Thu Sep 20, 2018 10:18 pm

I would just dump CS5 and go with CC 2017. It'll prolly fix all the issues you are having. Also, may want to use PTGUI 10.0.16. Those 2 program versions seem to be the most stable for live panos.

As for this question
This must also be the reason, why f.e. spoken text is not synchron with movement of the mouth ... Did this happen to anyone else ?
I am not sure about the missing frames but I know for a fact that after I have converted the sequence images into a video file that I have to add that video file into Premiere Pro along with the audio and the audio is always shorter than the video file so I have to stretch the audio file to match the video file otherwise the audio and video are out of sync.
Tawrank
Posts: 12
Joined: Fri Feb 15, 2019 9:28 pm

hey, thanks, i will go on next week ...
MajPG
Posts: 11
Joined: Sat Jun 20, 2020 5:11 pm

1uv3 wrote: Thu Feb 21, 2019 11:48 pm Hi,

Quicktime could definitely be the culprit. I am going to pm you.

I found that only Photoshop CC 2017 works for the live pano process. I am using Windows version. For some reason, other Photoshops wouldn't even allow me to import videos for image sequencing and are also missing the check box that enables you to import the image sequence towards the final steps.
Hi everyone!

I still have the exact same issue and can tell you that it's not because of the Photoshop Version (mine is CC 2020 21.1.1, Pano2VR is 6.1.8 pro 64bit Cocoa).
I did exactly what you did here:
https://www.youtube.com/watch?v=j6AZyEv ... e=youtu.be
and can still see the edges of the exported/imported video (patch) in my panorama (on MacOS!, on iOS it‘s fine).
If you get really close to the edges you can see that it might be a mixture between distinctions in color and image quality (visible pixels or mush in video).
It's not a project killer for me, but a beauty issue nevertheless.
What I noticed was that if you change the perspective in the panorama (e.g. you drag and drop your way slightly up/down or to one side), the edges of the video-patch appear / disappear sporadically (depending on your perspective) - which makes the issue even more conspicuous.

Did anyone (who had the same issue) find a way around yet?

Kind regards and stay healthy!
KrisMaz
Posts: 22
Joined: Mon Aug 03, 2009 7:43 pm

Hello, Everyone!

Just adding momentum to this thread...

I'm experiencing the same trouble with seeing the edges of a video patch. I'm hoping someone who has had even a little success with minimizing the appearance of the patch edges would share their insights.

I have included a couple screen captures to show where the patch is placed in Pano2VR and how it appears on Firefox when posted online.

Many thanks...

Take care and be well!
Attachments
Pano2VR Patch Area.JPG
Pano2VR Patch Area.JPG (78.34 KiB) Viewed 4510 times
Pano2VR Patch Area Online Screen Capture.JPG
Pano2VR Patch Area Online Screen Capture.JPG (306.16 KiB) Viewed 4510 times
User avatar
Hopki
Gnome
Posts: 13005
Joined: Thu Jan 10, 2008 3:16 pm
Location: Layer de la Haye, Essex UK
Contact:

Hi,
This is to do with the browser and the difference between a picture and a video.
What you can do is add an image mask which will reduce the problem.
Have a look at this example, use the mask button to toggle it on and off to see the difference:
https://support.ggno.me/forum_projects/bbq/
Regards,
Garden Gnome Support
If you send an e-mail to support please send a link to the forum post for reference.
support@ggnome.com
https://ggnome.com/wiki/documentation/
KrisMaz
Posts: 22
Joined: Mon Aug 03, 2009 7:43 pm

Thanks for getting back to me, Hopki! It helped a lot!

Following the process you documented when you created the BBQ example did not completely work for me. I found that PNG image patch edges have the same problem that the video patch edges have. See attached "PNG Patch Edges" screen capture. This was created by extracting a patch to Photoshop, saving it as a PNG, and then bringing it back into Pano2VR as an pinned image. The edges show the same way the pinned video edges.

What I did to get around this was instead of duplicating the patch area in list view (Step #2 in your documentation of the BBQ example), I created a new patch larger than the video patch area so that I could gradate the PNG image's edges with transparency. Attached are the video and image patch areas from Pano2VR. Also, included is a screen capture of how it looks online.

I'll be adding some adjustment layers to help further with the blending.

I'm so relieved to see the hard edges disappear. Thanks again for the prompt response and the solution it helped me find!
Attachments
PNG Patch Edges.jpg
PNG Patch Edges.jpg (296.79 KiB) Viewed 4489 times
Video Patch Area in Pano2VR.JPG
Video Patch Area in Pano2VR.JPG (92.44 KiB) Viewed 4489 times
Image Patch Area in Pano2VR.JPG
Image Patch Area in Pano2VR.JPG (99.58 KiB) Viewed 4489 times
Video Patch with Larger PNG Mask Image.JPG
Video Patch with Larger PNG Mask Image.JPG (91.96 KiB) Viewed 4489 times
Post Reply