Right now I am completing a set of maps covering from Dunedin to Mosgiel, some 17 km of rail corridor continuously mapped. The result is a Gimp file of over 60 GB in size and Gimp itself is using about 150 GB of storage (memory plus swap plus home drive cache) to allow this file to be worked on.
Previously I have written about how we can use bigger files by having sufficient SSD storage capacity, and pitched for bigger SSDs to enable larger projects to be handled. However the key problem with a big project is how long it takes for it to be saved to disk – over an hour with a file of this size.
So I am looking for a balance where I can have somewhat smaller project files that don’t take such a long time to save, and yet can still contain several station sites in one file instead of just one. This has been really an experiment, and only possible since Gimp 2.10 was released, because previous versions could not write more than 4 GiB to a disk file.
Prior to this Dunedin to Mosgiel project, the largest one I had on file is the OtiriaNorth file that covers some stations on the Opua and Okaihau branches. There is not as much stuff in that file by any means and it last saved at 17.8 GiB. There is not really a lot of merit in allowing this Dunedin-Mosgiel project file with 178 layers to have grown as large as it has. The long save time is really tedious, as is the resource usage in the computer. However I was interested in seeing just what Gimp could handle.
In a previous post I compared longitudinal canvas with a sectioned canvas with multiple overlays. I am starting to think a longitudinal canvas may work as long as the number of layers is kept low, so I will be experimenting with this format in Christchurch and see how long I can make the canvas before it gobbles up too much resources. In future at any rate I will work to 100 or fewer layers for files as a rough rule of thumb. I don’t think I need a bigger SSD anymore. But having the 95 GB of SSD available for Gimp has been very good.
The Dunedin-Mosgiel project has now been cut into two files as Gimp is having difficulty working with the large project size and has crashed a couple of times so as I still have a little work needed on the project then making it work properly with Gimp is a priority.