2800 nodes project compile black images

Q&A about the latest versions
Post Reply
Bostjan
Posts: 421
Joined: Fri Oct 17, 2008 4:06 pm
Location: Slovenia, EU
Contact:

I have a project with 2800 nodes which program loads well and the scratch is 10 TB (7200 rpm). It takes a few hours to compile but there it is an issue as already at the ~ 720th node the stored images are black (images and as well tiles). Not always exactly, but it varies from 703 to 728 node when the saved images are black (I tried several times to compile the project).
I believe that the graphic station is with enough "power" to compile such a project (I am adding even additional nodes in the future), the HDD might be a problem but I couldn't get the SSD of the same size at my place, so I am wondering where it is the catch to not compile all images. (?)
Bostjan
Posts: 421
Joined: Fri Oct 17, 2008 4:06 pm
Location: Slovenia, EU
Contact:

The example of the first 700 nodes which compiled well: https://www.burger.si/GorskiSvet/Julian ... tapa4.html - that is about 25% of the 280-300 km long hiking tour.
User avatar
Hopki
Gnome
Posts: 13005
Joined: Thu Jan 10, 2008 3:16 pm
Location: Layer de la Haye, Essex UK
Contact:

Hi Bostjan,
This is most probably the image cache location/SSD drive running out of space.
What you can do is delete the tiles folders that have the issue leaving the good ones.
The next output should skip the already remapped images and only start working on the remaining.

What will also help is if you convert the input images to TIFF using Pano2VR, this will convert your input images to pyramid TIFFs which will not need to be cached.

This will also speed up opening the project.
Regards,
Garden Gnome Support
If you send an e-mail to support please send a link to the forum post for reference.
support@ggnome.com
https://ggnome.com/wiki/documentation/
Bostjan
Posts: 421
Joined: Fri Oct 17, 2008 4:06 pm
Location: Slovenia, EU
Contact:

Tx. Martin,
It seems that I will need to upgrade the HHD from 10 TB to 16TB (the largest scratch one I can get right now). I have solved the part of the problem by removing the first 700 nodes and then recompiled again and merged the new set of directories with tiles into the previous output directory with the first set of tiles directories.
Neal
Posts: 193
Joined: Thu Dec 12, 2013 11:14 am
Contact:

Instead of upgrading to a 16GB drive, looking into adding a second (or third) 10GB drive. Then use either spanning the drive volume across the 2 drives or use MKlink to create a hard link to the second drive. The hard link makes the second drive look like a directory of the first drive.

here are some links on how to do each:
https://www.windowscentral.com/how-crea ... windows-10
https://www.educative.io/edpresso/mklink-windows-10
User avatar
360Texas
Moderator
Posts: 3684
Joined: Sat Sep 09, 2006 6:06 pm
Location: Fort Worth, Texas USA
Contact:

Morning Neal.

I think Bostjan said 16TB. Will the MkLink function still work with the larger drive ? After reading... guess it does not matter.
Dave
Pano2VR Forum Global Moderator
Image
Visit 360texas.com
Neal
Posts: 193
Joined: Thu Dec 12, 2013 11:14 am
Contact:

For those interested, the mklink works on any drive. The biggest advantage is that it looks like a folder and is treated by Windows as a folder. So you can use it to move necessary files off your OS drive and on to a second drive. This includes hard to move folders like the "Program Files" folder or even the %appdata% folder.
User avatar
Hopki
Gnome
Posts: 13005
Joined: Thu Jan 10, 2008 3:16 pm
Location: Layer de la Haye, Essex UK
Contact:

Hi Bostjan,
If you make yourself a droplet to convert the images to TIFF, then you won't need to add any more space.

To make the Droplet
Open a new project, with no input images add a Transformation Output.

Settings

1.png
1.png (144.28 KiB) Viewed 3260 times

Note: Format: TIFF (tif) and Output File: output/$(n).tif

Then select File => Save as droplet then don't bother with any of the settings in the next window.

2.png
2.png (31.88 KiB) Viewed 3260 times

Click Create and save to your desktop.

Drop a folder full of images on this and it will convert them to pyramid TIFF's.
If you use these in a project, 1) it will open in seconds, 2) will not require much image cache space.
Regards,
Garden Gnome Support
If you send an e-mail to support please send a link to the forum post for reference.
support@ggnome.com
https://ggnome.com/wiki/documentation/
Bostjan
Posts: 421
Joined: Fri Oct 17, 2008 4:06 pm
Location: Slovenia, EU
Contact:

ok will try the droplet.

GB ... sure TB that was a lapsus :)
User avatar
Hopki
Gnome
Posts: 13005
Joined: Thu Jan 10, 2008 3:16 pm
Location: Layer de la Haye, Essex UK
Contact:

Hi,
Just to put things into perspective.
A folder containing 2,240 TIFF panoramas had a size of 997.86GB.
When loaded in Pano2VR the image cache file size was 1.49TB

Made a droplet and converted all the images to Gnome Pymiried TIFF, each image took 6 seconds to remap and save, the resulting image folder was 1.04TB which is to be expected as each new TIFF has all the multi-resolution data.

Opened a new project, added all the new images, which took seconds to load.
Image Cache Size. 0KB

Regards,
Garden Gnome Support
If you send an e-mail to support please send a link to the forum post for reference.
support@ggnome.com
https://ggnome.com/wiki/documentation/
Neal
Posts: 193
Joined: Thu Dec 12, 2013 11:14 am
Contact:

The Gnome Pymiried TIFF, is that the same as saving a TIFF in Photoshop with the save Pyramid open selected? Also, what was the size of the output folder on that 2240 project?
User avatar
Hopki
Gnome
Posts: 13005
Joined: Thu Jan 10, 2008 3:16 pm
Location: Layer de la Haye, Essex UK
Contact:

I say Gnome Pyramid TIFF, it's just a Pyramid TIFF but converted using Pano2VR.
The Output Folders would be the same size.
Photoshops Pyramid TIFF should be the same, but when I tested it was slower.

Real-time 2240 nodes loading.

ScreenFlow.gif
ScreenFlow.gif (1.83 MiB) Viewed 3223 times

I know I point to the Adobe RGB, but I was referring to the image size.

When loaded the project is ready to work on.
I did not make the output as I was concentrating the image cache issue, which if you use the droplet to convert the images there is none.
But following your post, I dropped one of each, normal TIFF and Pyramid TIFF and both tiles folders were 24.2MB each, which would be expected.
So the output would be about 54.2GB
A small 2TB SSD should be plenty.
Regards,
Garden Gnome Support
If you send an e-mail to support please send a link to the forum post for reference.
support@ggnome.com
https://ggnome.com/wiki/documentation/
Post Reply