Page 1 of 1

Generating tour on external NAS disk is still very sloooooow

Posted: Thu Mar 15, 2018 5:55 pm
by skyfocus
Hi,

I've installed the new version 6 beta today and I was very curious if there was a fix for the very slow performance that I still have since version 5 when generating a virtual tour when files are located on a Synology NAS. I did report this also around 2015 when 5.0 came out for the first time.

It is no problem when a tour is only 5 nodes big but when generating a tour that is 100+ nodes I can drink a lot of coffee before it finally finishes. When copying the files to a local disk it all goes very fast. But copying the data back and forth from my local machine to my NAS is very annoying and prune to errors, in the end you don't know which version is the latest anymore. The final tour is a bummer to copy too because of the gigantic amount of files that have to be copied.
You can have a very fast network but that will not help when the amount of files is very very big, the file handling becomes an issue.

I use a Synology DS1812+ 8 bay NAS connected on a 1Gb network (all adapters are 1Gb). The max throughput I have seen while copying files is around 100MB/s ....

Just to make clear where I'm talking about I did a test with a medium sized tour of 14 nodes located on my NAS and the same tour
located on my local drive:

Synology NAS = 9:25
Local disk = 0:35

So when the tour is 140 nodes it will take 1,5 hour to complete and the local version will finish in 3.5 minutes !!!

I've asked a friend to generate the same tour on his network storage (a dedicated Linux NAS), he reports the same performance.

Hopefully this can be fixed in the final 6 version .....

Regards,

Arjan

Re: Generating tour on external NAS disk is still very sloooooow

Posted: Sat Mar 17, 2018 11:13 pm
by Hopki
Hi Arjan,
I have given the devs a link to this post.
But whats the speed of your internal drives?
I use an external SSD using USB 3.0 which uses up to 5GB second which is quite quick.
My internal SSD speed is 6GB.
USB 3.1 is up to 10GB sec so NAS using 1GB is very slow in comparison.
Be interesting to see what the dev's say.
Regards,
Hopki

Re: Generating tour on external NAS disk is still very sloooooow

Posted: Mon Mar 19, 2018 8:00 pm
by Hopki
HI Arjan,
I have spoken with Thomas about this.
The problem is latency not so much the network speed.
He has this on his radar but that said I would not do this myself.
Better to keep it local and back up/transfer to NAS when done.
Regards,
Hopki

Re: Generating tour on external NAS disk is still very sloooooow

Posted: Tue Mar 20, 2018 10:06 am
by skyfocus
Hi Hopki,

The local drive I've tested on is an internal 500Gb SSD.

I also do think it is latency that adds up with the amount off small files and not network performance. I did do some tests and you can clearly see that disk write/read performance is cranking op with files bigger than 256KB, anything lower than that will have a big performance impact.

Image

The differences in disk performance between the NAS and the local SSD with the smallest file size is a factor 20, almost the 25x performance difference .....

Copying the files to my local NAS is a workaround that I'm using now but it is not the optimal situation at all.

Because
1) I'm working from both my 2 travel laptops and my PC on the same projects. When copying the project to my local PC disk it is not accessible on my laptops anymore and visa versa.
2) I'm also backing up my files located on my NAS, copying the data to my local machine means that the latest changes will not be backed up anymore.
3) Keeping track of the latest versions/changes becomes a big pain when working on 50+ projects ...
4) I don't know if you tried copying the output of a 140+ node tour, the same problem with latency arises here because of the amount of files to be copied. I keep a copy of all tours I have online on my webserver (allmost a 1000 virtual tours) on my laptop for presentations. It takes almost 2 weeks to copy it from my webserver and to total amount of data is around 300Gb ! My internet connection is 400Mbits/s .....

It would be nice when this issue can be addressed, maybe it is possible to have all those small files in an image-container like it was handled with the flash format.

Regards,

Arjan

Re: Generating tour on external NAS disk is still very sloooooow

Posted: Tue Mar 20, 2018 6:09 pm
by Hopki
Hi Arjan,
Had an idea, change the tile size as found under the Image tab of the HTML5 output.
This is set to 510px but as we add 1px overlap the actual size is 512px which is the optimal tile size for graphics.
However if you set this to 766px this should generate tiles over 256KB spending up the file transfer.

In fact if you use a constant input image size, example 12K then you can then manually add levels and work out a tile size that will exactly fit in the the cube face eliminating odd sized tiles.
To make this easier to calculate I would make sure that Use width/Pi as output factor is deselected as found under settings/preferences and under the Advanced tab.

12000 / 4 = 3000K
Manually add three levels
3000
1500
750

Set the tile size to 750px

Every tile should be 752 x 752px each at around 637KB
You can then save this as an output template.
Please test as the should improve speeds.
Regards,
Hopki

Re: Generating tour on external NAS disk is still very sloooooow

Posted: Tue Apr 10, 2018 10:05 am
by skyfocus
hmm, ok, that might be a solution for future tours, I'll try it and report back if it will help.

I don't think this is the path to go for older tours because I have to regenerate every tour with new tiles, trowing away the old tiles, with 1000+ tours it will take a full year to do so .... if ... it will make it faster ... I' will come back on that.

regards,

Arjan

Re: Generating tour on external NAS disk is still very sloooooow

Posted: Fri Apr 13, 2018 8:25 am
by mbb
I also use a Synology NAS (DS918+) and have not noticed any unusual slowness when working with my tours or their files (either with Pano2VR or even Panotour Pro). My NAS is connected via Gigabit Ethernet to my router, then Gigabit Ethernet from my router to my Windows10 laptop. Files are accessed on my Symology via mapped network drive in Win10.

If your NAS is utilizing an additional internal SSD, or extra RAM cache (my Synology has an SSD cache option but I do not use it), that might be the cause of the (additional/unusual) latency? .

EDIT:

I've gone and regenerated one of my VR Tours and timed it-

11 nodes, each from 10,500 x 5,250 equirectangular source images, multi-res, w/512 pixel tiles. All files located on the NAS.

Total time for regenerating tiles was exactly 2 minutes. Which I think is reasonable.

You may want to go over your Synology's settings for network, performance etc. if you are getting times greatly exceeding this?

Re: Generating tour on external NAS disk is still very sloooooow

Posted: Fri Apr 13, 2018 9:00 am
by mbb
Additional notes regarding the many small tiles utilized by these types of vr tours.

First of all, I have about 10,000+ tiled vr panoramas myself, that all get shuffled around to one place or another- so I feel your pain ;)

Latency is indeed increased because of the many small files. It is the way of things. Over local SSD, HD or networks, it doesn't matter.

When copying (sFTP) to your remote web server, I would highly recommend TAR'ing the files in "chunks" before transfer. -Provided you have shell access on the server so you can un-TAR again.

The real nightmare is when transferring the files to cloud storage services like AWS S3, Digital Ocean, DreamObjects etc. Not only is transfer speed greatly reduced because of the per-file protocols used, but many services like S3 charge a fee *per* PUT/GET request. So in effect you are getting dinged for each of those little tiles (no native TAR/UnTAR capabilities with cloud storage btw). Therefore, reducing the number of tiles by using larger sizes can have quite an effect on reducing your total upload times and cloud fees.