Pinning Video

Q&A about the latest versions
paulk
Posts: 90
Joined: Thu Apr 29, 2010 10:53 am

Hello all
Thought I'd post this and see what feedback I get.

I've looked at this great 'In the wild' post https://ggnome.com/2021/07/pano2vr-in-the-wild-circus/ and was really impressed with the results
I've also looked at the methodology as described here: https://ggnome.com/doc/embed-video/
My question here is does anyone have a method to achieve this using the same lens that the panorama is shot on?

For example I'm currently using a Nikon z7 2 with a sigma 8mm fisheye. In an ideal world I'd like to shoot the video using the same lens, during the panorama shoot.
For example - shoot the panorama (say 8 shots at 45°). Then shoot a video like in the 'In the wild' example above, say at position 8 - that way the camera / tripod is exactly in the right position and all I have to do is switch from stills to video on the camera.

The problem I'm having is how to convert the fisheye video so it can be pinned as per the Garden Gnome instructions.
Trying to 'De-Fish' the image is proving to be very difficult.

Anyone out there who has any suggestions that would be great!

Look forward to your replies.
User avatar
Taka
Posts: 51
Joined: Fri May 28, 2021 3:15 pm
Location: Tokyo

Hello, paulk.

Do you mean you want to convert 360 degree panoramic video to straight video?

I could only imagine attaching an iPhone to a tripod, or putting a camera on top of a camera. :(


By the way, I use GoProMAX, so it's probably no problem to shoot as it is.


I also want to make a great tour like Circus.
paulk
Posts: 90
Joined: Thu Apr 29, 2010 10:53 am

Hello Taka

No, not converting an entire 360° equi video - there's a lot on how to do that out there - in fact that's the method 3dVista use in their how tos.
What I'm taking about it is this sort of scenario:
A client wants a 360 of say a reception area, which we all could probably do quite easily. But they want a member of staff standing there and when the relevant button is clicked/pressed the person becomes animated (video) and says a few words etc.
Exactly what is shown in the Circus 360's.

In a normal working enviroment, like mine, I've the camera/lens already set up for the 360. It makes sense to film the animated part using the same lens set up - and without moving the tripod.
So what's needed is a methodology to use the fisheye video and the pin it like the example Hopki showed in his tutorial.
The diffiuculty is straightening the distoritions in a fisheye video so it can be pinned as per the tutorial.

I'm investigating a number of options availiable in photoshop, After Effects and Premiere Pro.
So I'll let everyone know what I discover.
In the meantime it would be great to here from everyone as to how they'd approach this. Maybe Hopki could chip in too!

Paul
User avatar
panox
Posts: 305
Joined: Mon Mar 12, 2007 11:03 pm
Contact:

Hi Paul,

I tried that with Sigma 8mm-lens but the results were not satisfying. You have to warp the fisheye-lens-video to a rectangular shape. I had best results with videos of my Osmo-Pocket.
User avatar
Hopki
Gnome
Posts: 13004
Joined: Thu Jan 10, 2008 3:16 pm
Location: Layer de la Haye, Essex UK
Contact:

Hi,
Think about how you embed video.
First you extract a patch image, this has very little distortion compared to a fisheye video.
You are better taking the video with a mobile phone, most are HD so that's all you need.
Also the video should not be larger than 1024 x 1024px if you want it to work with mobiles.
Regards,
Garden Gnome Support
If you send an e-mail to support please send a link to the forum post for reference.
support@ggnome.com
https://ggnome.com/wiki/documentation/
paulk
Posts: 90
Joined: Thu Apr 29, 2010 10:53 am

Hi Hopki
Thank so much you for adding your comments.
I think what would be very useful, as I'm sure there's a lot of Pano2VR users who would like this, is a overview of the techinical issues here.

Not so much what device the video is shot on or necessararliy the method used, as this will be very differrent dependant on the paying client / senario (using mobile phones on a professional shoot is a good example).
What I was thinking was the parameters in play here. i.e. video size, resolution, format, browser issues, best practice etc.
Although you covered a lot in your webinar this is quite a big subject in terms of complexity and actual real world use, especially as I'm sure a lot of fellow professionals would like to produce work as highlighted in the Circus example - think we all need a good set of guidelines, in the same way we all approached changing to HTML5 from Flash (those were the days!).
I would be interested to hear your thoughts on this.

Many thanks
Paul
User avatar
Hopki
Gnome
Posts: 13004
Joined: Thu Jan 10, 2008 3:16 pm
Location: Layer de la Haye, Essex UK
Contact:

Hi Paul,
With regards to WebXR, Pano2VR supports node hotspots as well as pinned video.
Our WebXR also supports stare navigation as well as wand interaction, so you can click on the pinned video to play it etc.
I know "other" players only allow stare navigation.
But never felt it was a subject worth covering in a webinar with its limitations.
This will change in the next major release so I expect it will get so air time in a webinar.
Regards,
Garden Gnome Support
If you send an e-mail to support please send a link to the forum post for reference.
support@ggnome.com
https://ggnome.com/wiki/documentation/
paulk
Posts: 90
Joined: Thu Apr 29, 2010 10:53 am

Hi Hopki
Think some wires got crossed there - I wasn't referring to WebXR (I've never had the need to get into it myself). I was talking about pinning video like the Circus example.
I was looking at a set of guidelines on pinning video - sizes, formats etc, as I mentioned in my last post.
User avatar
Hopki
Gnome
Posts: 13004
Joined: Thu Jan 10, 2008 3:16 pm
Location: Layer de la Haye, Essex UK
Contact:

Hi Paul,
Yeah, having a brain fart.
video size, resolution, format, browser issues, best practice etc.
If you want the video to work on all devices then keep it small, Apple state you should not go past 1024 x 1024 for media, this also includes pinned images.
But I would say Android is worse, not seen anything about size or resolution for Android but have had projects work on iOS and not Android.
Reducing the resolution and bingo, all starts to work again.
So I would say do not go above 1024 for the longest edge on the understanding that videos are not normally square.

Format, MP4, when encoding the video make sure you use something like HandBrake that has the Web Optimise option, this puts all the data of the video in its headers which mean sit can stream. You will notice some video have to download a good part before they will start to play, these are not web optimised.

Currently the last Safari update broke pinned video, as a test we made a two node project with node two having a pinned video, as you switched back and forth between the nodes you would see the video and then not, but always hear the sound.
So randomly you do not see the video but always hear it.
We are looking into this, but this happened with Safari 15, all other browser are OK.

The Ellison project that we showed in our blog used video resolutions of 944 x 944px with AC3, ACC, H624.
This looked amazing and still within Apples limit.
Regards,
Garden Gnome Support
If you send an e-mail to support please send a link to the forum post for reference.
support@ggnome.com
https://ggnome.com/wiki/documentation/
paulk
Posts: 90
Joined: Thu Apr 29, 2010 10:53 am

Hi Hopki
As usual a thorough reply.
I'm about to start a major project in the next month or so and I want to incorporate this type of thing, as I think it can really be useful and maybe replace some of the inevitable pop up boxes etc.
I’m thinking that the Ellison project may have used rotoscoped videos as the backgrounds are very well blended and there’s a hint of shadow under the feet of the presenters.
So I’m going to do two versions one using your method, with the parameters you’ve suggested and another in front of a green screen (which will be rotoscoped).

Also, I’ve tried to ‘flatten out’ the fisheye video and tbh it’s very fiddly and the results are mixed at best.
So I’m going to change lenses once the 360 is shot use a 12mm zero distortion wide angle for the video portion and see how it goes.
Wish me luck! :D
User avatar
Hopki
Gnome
Posts: 13004
Joined: Thu Jan 10, 2008 3:16 pm
Location: Layer de la Haye, Essex UK
Contact:

:D
Would be interested in your journey, note the codec AC3, this was for Safari.
Without it, the video and sound were out of sync.
Regards,
Garden Gnome Support
If you send an e-mail to support please send a link to the forum post for reference.
support@ggnome.com
https://ggnome.com/wiki/documentation/
paulk
Posts: 90
Joined: Thu Apr 29, 2010 10:53 am

Hi Hopki
I will certainly share the experience once I've published it.
As a matter of interest is there any way to set the pixel size of a patch in the project? or is it a bit of guesswork - i.e. fov v aspect ratio etc.
Would be useful so not to fall foul of the imposed size limit.

Btw I've looked at the Circus 360 again and alas it's not working on Safari mobile version 15 :(
User avatar
Hopki
Gnome
Posts: 13004
Joined: Thu Jan 10, 2008 3:16 pm
Location: Layer de la Haye, Essex UK
Contact:

Hi,
Hummm, as your extract an area from the panorama you just use FoV.
Why pinned or embedded video is not working was down to Apple.
Apple had a bug in iOS which iOS 15.1 fixes, however, desktop Safari still has it and are waiting for the fix from Apple.
Regards,
Garden Gnome Support
If you send an e-mail to support please send a link to the forum post for reference.
support@ggnome.com
https://ggnome.com/wiki/documentation/
User avatar
Tony
Posts: 1341
Joined: Mon Feb 15, 2010 6:54 am
Location: Adelaide, South Australia
Contact:

Hi Martin,
Apple had a bug in iOS which iOS 15.1 fixes, however, desktop Safari still has it and are waiting for the fix from Apple.
Running into some problems with pinned video not playing in Safari, did Apple ever fix this?

Tony
Tony Redhead | Panoramic Photographer | mobile: +61438501002 | website: https://tonyredhead.com - https://redsquare.com | Pano2VR Tutorials: https://tonyredhead.com/pano2vr | instagram: https://www.instagram.com/tonyredhead/
User avatar
Hopki
Gnome
Posts: 13004
Joined: Thu Jan 10, 2008 3:16 pm
Location: Layer de la Haye, Essex UK
Contact:

Hi Tony,
Testing your project, it works for me on Desktop, iOS, iPhone11 pro iOS 15.3.1, and Android 12.
However, in V7 there is an issue with polygon hotspots and z-index on the desktop.
Changing the project to compensate for this and it all works as expected.
Have you tested in both Pano2VR 6 and 7?
Regards,
Garden Gnome Support
If you send an e-mail to support please send a link to the forum post for reference.
support@ggnome.com
https://ggnome.com/wiki/documentation/
Post Reply