Program Crash Destroyed File

[alexart]

I was working on an illustration (430mm x 307mm, 400dpi) with multiple layers. I know it's pretty big, but it is for print and I am professional about my content. Artrage started showing "string table error" message over every command.
At this point, I realized I hadn't saved in over one hour and tried to save the document, and that turned up the error message again (and something about not being able to write, I can't remember very well). When the program finally crashed and i tried to find my file, there was a document of 5.09KB in its place. That couldn't be the file i was working on. I tried to open it, Artrage said something about recovery (I enabled Backups and Temp files to be created) but that failed.

The document was big (before the crash), 250MB file size and had around 30 layers. The backup files I found after the crash were only partial files, the biggest one was only 100MB and it had just some of the layers i was working on. I could not patch the original file from the backup files' layers. Maybe if the backup file was an exact copy of the previous file, this wouldn't be a problem.

When I work i keep the Windows Task Manager open to see what my programs are doing. AR was using 1.88GB of RAM, out of 8 total. Total RAM usage was under 5.5GB at that time.
My setup is Win7 64bit, Q9650 3gHz, no overclock, 8GB of RAM, Quadro FX1800, Cintiq 2100dtz as secondary display and input surface. All drivers are up to date. All system updates are in place. AR is the latest version (4.0.2). HDDs were checked for errors and bad sectors (none found) and are new.

Will this keep happening? What are the limits for AR on my setup? Can you tell me some way to safely work on such big files? I got used to AR being slow on big files, but this is the first totally destroyed.
Could a new backup system help this problem? When the user presses "Save" the file gets a name change (to "File_Backup.ptg") and the new file is written from scratch on the hard disk. This way, the old file wouldn't be changed if the program crashes and it wouldn't be destroyed.

Thanks for listening.

[MattRage]

Sorry to hear about the crash, let's see if we can work out what's happening:

The strings in the interface shouldn't be able change to string table error without the application being restarted - They are set at load time and not touched after, so it sounds like something caused a problem with your install files between ArtRage closing and being reopened. Did all of the strings change, or did you just notice that menus you had not opened before were showing the error? Do the strings appear as string table errors now if you launch the application?

The file you're working on isn't huge, so while there's a chance there could have been a memory problem it doesn't seem likely. Unfortunately without the exact error message I'm not sure what happened. If you do see anything like this again, please make sure to report the error online when it happens: If prompted, opt to search online for a solution then when that finishes, you should be asked if you want to submit a report. Choose to do so and we will get that report which lets us track down what's going on in the application at the time of the crash.

ArtRage 4 does contain a backup system accessible from the preferences panel (under advanced options). Backup copies created by ArtRage 4are exact copies of the previous file, here's how it works: When you have the backup system turned on, when you Save, ArtRage renames the old saved version of the file (it doesn't do anything to that file's contents) and then saves your new version. The recovery file is a file ArtRage attempts to save to a temporary location if it detects a crash. In some cases the file may not be able to be saved properly and this may be why you had problems recovering.

[alexart]

Thank you very much for your response.

I think I may have made an error explaining the situation: the error message wasn't appearing on the menus, but whenever i would click a command or brush, the error would pop up in the middle of the screen (including when trying to save the file). I am not sure what the strings you are talking about do or are. I think they're above my level of understanding.

I do however have an issue with the menus: sometimes, after a large layer copy or group move in the layer hierarchy, the progress bar disappears and when the task is finished (10, 15, 30 seconds) some menus disappear from the screen (for example the layers menu is disappearing and the layers smaller button on the side is also gone) and i have to go into workbench mode and back to show the menu again. I am not sure that this is what you meant or if it is a serious thing (I don't have a problem with this, it's minor). The program works fine and there are no display issues when just starting up or on small files. Only when the files get bigger the minor display problems appear. One thing i noticed, the program hogs all the processor cores while doing some operations (save, image adjustments, layer commands) and all other programs start to lag a little (music stutters, video starts showing tearing artifacts etc.), probably HDD or processor latency, not sure. I usually set the program affinity (in task manager) to 3/4 cores or even 2/4 and the problem is gone.

The backup system you explained to me is ideal. I tested it just now, and the files were behaving perfectly. However, the file that got shredded had backups that only contained some of the layers (suspiciously, only a few layers modified just before the crash). I'm only saying all this as info, not a rant. Artrage IS fantastic.

I will reinstall the program, in hopes of never seeing that error again. If it will come back I will report the issue as you described. Last time I panicked a little, to be honest.

A few last questions (I promise I won't burden you further): what is the maximum RAM that AR can use? Is it maxed out at 2GB, as a 32bit application? If there is a limit, can I still work on files that take up more than 2GB of RAM?

Thank you so much for your patience!

[alexart]

Happened again... After reinstall.

I almost finished redrawing the file that got lost yesterday. The backups were all doing fine, all the right size and all that.
I was adding the final touches and I got a hunch and made a copy of the file before hitting "save". Finished the drawing, hit save and then:

"An error occurred on attempting to read/write data from/to a file stream.
Artrage encountered an error trying to save the requested file. You may need to restart the application."

I look at the file size in explorer: 5.10KB (instead of 273MB). I try to close the program, as "save" wasn't doing anything but showing the error splash again. When I tried to close the window from the "x" this happens:

"Not enough memory to perform action."

I try again, same result. I wait for the final crash to happen but the program just hangs for 3 minutes. I use Task Manager to end process.
I try to open the 5.10KB file and hope that AR could recover anything from the backups and temp files. This happens:

"An error occurred on attempting to read/write data from/to a file stream.
File = XXXXXXXXXX Layer = 0"

I now notice that the backup file generation A, the last one created had become 5.10KB too, thus destroying the last remainder of my file. The older backups are still intact, but if i try again, the problem would probably just move further down, destroying all the backups.

I'm very sorry that I couldn't use the Send Report utility, as I had to end the process and the dialog never showed up. I tried to document the whole incident, and it was a replica of the first one reported. The only difference was that I only tried the commands once, avoiding a chain reaction in the backup files.

After this crash and file's effective destruction, i opened the backup i made manually, did my final touches (just a few brush strokes with sticker spray) and the file saved ok, backup creation and all (file save at this size usually takes 1-2minutes, i don't know if it's normal, but it was always this slow for me). The difference was that this file was only opened for a few minutes and the only commands were the few brushes and "save". I wanted to xport the file to PSD but it didn't show the dialog for that, so i closed the program, reopened the file and export worked. The other file, the one compromised, was open for cca 30 minutes (maybe one hour) and had a lot of history of commands and brushes. RAM usage for the two was basically the same, 1.8?? GB (so, I think most of it was bitmap data).

Well, this is the new info, maybe it can help somehow. Again, sorry for not being able to complete the "send report" steps.

[MattRage]

Thanks for the clarification. It does sound like a memory error though the appearance of a string table error in a panel is really odd because the strings are read in at application launch time, when memory hasn't been used for anything else. It's possible that that is a general error caused by something else, more on that in a moment...

Regarding memory: ArtRage can access around 3.8Gb of RAM as a 32 bit application. While the file size in general is fine, it may be that the amount of layers you have with the amount of content is causing a memory error. In general, ArtRage 4 uses slightly less memory than 3 and handles layers differently to minimise the possibility of crashes, so this does sound like an unusual case. Do you have lots of references or stencils? If you have references, how big were the source images?

The backup system maintains previous backups so if you had a corrupted file that was not backed up, loaded your copy of the correct version and saved it as the original file name, the previous corrupted version would be shuffled down to the first generation of backup... Okay, that was confusing but the general gist is: Whenever you save a file, any previous versions of that file (including backups) are shuffled down the generation list to a maximum of however many generations you defined. So - The corrupt file becomes the GenA backup and the last successfully saved version becomes the GenB backup. I think that's what happened here and why your Gen A backup was corrupt: It was the file that failed to save properly, renamed when the new file was saved.

I'll have a chat with one of the other developers here to see if we can come up with anything specific. It does concern me that you're seeing a string table error. I'm going to check in to what kind of errors can be spawned when the tools are selected and see if there's anything I can see there that might be memory affected.

[alexart]

In the second incident the "string table error" never showed up. Maybe that was some sort of accident caused by the memory problem. I trust the second incident's DIY report more, because i didn't panic like the first time and I wrote everything down.

I don't use references or trace images.The files I usually work begin as an import of a PSD file with some layers as a start-up and at the resolution I explained yesterday (430mm x 307mm, 400dpi). After the import i start adding background elements, layer by layer. This usually builds up to 10, 20, 30 layers, very few of them have a blend setting (overlay, multiply etc.) and I group them as to move them around easily. I do use Transparency Lock a lot. I don't generally use canvas lighting or grains. I use almost all the brushes and tools.
I rarely use stencils and I never keep them hidden, I delete them after I'm done with them. I do use a lot of Sticker Spray, always with Auto-flattening enabled (I prefer to just redo the brush rather than modify it). My sticker brushes are, however pretty big. The stickers are made from PNG files that are pretty big (the largest is ~8000x9000 pixels and has 6 columns and 2 rows, the average files are ~3000x4000 pixels). My "Resources" folder is ~500MB (I combined all the resources i have in this folder, for easy replacement if i reinstall).

I understood how the files are backed up, and it does actually work as intended on my system. I think that the first file's backups got shredded because of migrating the problem to the older ones as I kept trying to save the corrupt file. But AR also couldn't recover the second crashed file, even if only one backup (GenA) was corrupt.

As an extra detail, I sometimes see the RAM usage for AR drop from 1.8GB to 0.6-1GB and then rebuild up to 1.8GB in 5-10 seconds. I am not sure if it's designed that way or if it tries to repair some stuff in doing so.

Thank you very much for your help and care!

[MattRage]

The shift in RAM you're seeing is probably coming from our new layer system. In order to optimise memory use, ArtRage drops hidden layers to a disk cache rather than keep them in memory. This is also done at save time to try and avoid situations where the save process needs to allocate more memory than is available due to the size of the painting.

The sticker spray may be causing the problem however. The PNG files used in the sticker sheet will be taking up lots of RAM - Could you try reducing the size of the sticker to see if that helps?

We're continuing to work on memory handling and I suspect that's what's causing the problem here but it's odd that memory never heads close to the 3.8Gb limit.

[alexart]

The RAM shitf sounds useful and I did see it happen when saving files too.
I will try to lower the resolution of the stickers, or maybe, break them up into smaller ones too.
I''ll also try to get AR to go up to 3.8GB of RAM, or at leas jump over the 2GB I seem to hit everytime (maybe it actually did go over it at one point, but i didn't notice, because no problems ocured).
I'll get back home in a day or so and I'll try these steps. I'll report back if problems persist.
Thank you very much for your answers and help.

[alexart]

OK, i have done what you said about the stickers. I even uninstalled AR4 from my computer and cleaned registry after it was gone, so no trace was left. I reinstalled and didn't use any of my resources, i just left that standard folder untouched.

I tried to open the file that I was working on, and, because it's pretty complex (273MB), it goes to 1.7-1.8 GB RAM occupancy from the start. I tried to do some doodles on it with the standard brushes (watercolor, sticker spray, oil brush etc.). After it got past 1.8 and even ~1.92 it showed me the "out of memory" error. This is the stage that caused all my save problems before. As soon as this baby shows up, I know I'm in trouble.

I tried to open a new document, just to eliminate the suspicions on my file. It open a standard 1600x1200pixels file. I re-sized it to my usual dimensions (400x300mm at 400dpi). It jumped from 100MB of RAM to 1.2-1.4GB RAM (I left the canvas grain and texture on). I started doodling with standard brushes and it ran "out of memory" even sooner, at 1.5GB RAM occupancy.

I then installed AR4 on my mobile setup and I replicated EVERYTHING, even the RAM usage was the same for every situation. With standard brushes and custom, old complex file and new from scratch.
My mobile setup is Acer Aspire 5750G (i5-2410M, Nvidia GT 540M, 8GB RAM), Win7 64bit and Cintiq 12WX as secondary display, all new drivers, and up to date system.
On my desktop I even went into BIOS and switched Memory Remap on and off, with the same result (just different memory available for Windows - 7.1GB and 7.4GB).
What I did notice is that bigger sticker brushes, multiple layers and larger canvases fill up RAM more quickly, but the limit (1.8GB) can be reached with normal brushes too.

I can't get AR to move past 1.8-1.9GB RAM occupancy. I am not sure what is going on here. Maybe some Windows updates messed something up, or some limitation is in place with AR. I even tried running it in Compatibility Mode as Win XP SP3.

Thanks for listening and I hope this info can help the team in some way. For now, I'll just look out for RAM usage and divide my work load carefully.

[alexart]

Extra step I tried:

I tried to get the application to crash and perhaps be able to send an Error Report but it just won't crash normally. After it gives the "out of memory warning", it just sits there and it keeps showing "out of memory" after each command. No matter how often I try a command, how fast I click, it won't crash. If I try to save the file, however, it empties RAM from 1.8-1.9GB to 0.6GB and then it starts filling it up again until it reaches the 1.8GB limit. And then it shows this:

"An error occurred on attempting to read/write data from/to a file stream.
Artrage encountered an error trying to save the requested file. You may need to restart the application."

I click on close window (x button), it asks if I want to save file, I click "no" because "save file" just returns the previous error. I restart AR, new file is opened, no recollection of the previous file. The file that crashed while saving is now 5.13KB, just like the usual shredded files.