Memory issues

Here you can discuss and share functionality improvements and helper programs to make Context Free better.

Moderators: MtnViewJohn, chris, mtnviewmark

Post Reply
User avatar
kipling
Posts: 91
Joined: Wed Jun 18, 2008 2:36 am

Memory issues

Post by kipling »

I think I have found a memory issue with my latest version of crystals (http://www.contextfreeart.org/gallery/view.php?id=3161).
I had suspected that there were some memory problems while I was putting together the Turing code, but hadn't been able to pin it down. With crystals, I had to restrict the render in various ways to get it to work, both in ContextFree.app and cfdg - far more so than the original. Quitting and restarting ContextFree.app seemed to help sometimes, which suggests a memory leak. (It also caused everything else on my machine to swap out, which was painful.)

Anyway this is from version 3.0.2 of the command-line cfdg under Mac OS X, which has run perfectly previously, but just this once threw a malloc error:

Code: Select all

cfdg -w 500 -x 0.5 -v CRYSTL -T 1x2 crystals-2013.cfdg crystals-2013-CRYSTL.png
Reading rules file crystals-2013.cfdg                            
Restarting as a version 3 design                            
Compiling 2nd phase                            
3 rules loaded                            
Generating 16bit color PNG image, variation CRYSTL...
500w x 866h pixel image.
Generating...
    924666 shapes - 31174 expansions to do         
Rendering...

Writing 500w x 866h pixel image...
    1111710 shapes                                    
DONE!
The output file name is crystals-2013-CRYSTL.png
cfdg(60656) malloc: *** error for object 0xa09a79e95d7e9fbb: pointer being freed was not allocated
*** set a breakpoint in malloc_error_break to debug
Abort
In the old version of crystals, I vaguely remember it could routinely handle 10^8's of shapes, but now it is struggling with 10^6. I had put it down to the increased complexity of the task - in the old crystals, the number of z's was O(sqrt(number of shapes)) - so it could handle ≈10^4. In the new one, the number of z's is O(number of shapes) - pretty much every shape has its own z.

However, now I am thinking that maybe there is a memory leak somewhere.

I haven't yet got a fresh version of command-line cfdg to experiment further, as the "user interface hangs" fixed in 3.0.4 may not apply here. If I get time I'll try this later today.

User avatar
MtnViewJohn
Site Admin
Posts: 882
Joined: Fri May 06, 2005 2:26 pm
Location: Mountain View, California
Contact:

Re: Memory issues

Post by MtnViewJohn »

I get a segfault with that command line. It works fine in Context Free.app. I will check it out in lldb and see what is up.

User avatar
MtnViewJohn
Site Admin
Posts: 882
Joined: Fri May 06, 2005 2:26 pm
Location: Mountain View, California
Contact:

Re: Memory issues

Post by MtnViewJohn »

On line 66 of src-common/abstractPngCanvas.cpp change

Code: Select all

    mData = new unsigned char[mStride * height];
to

Code: Select all

    mData = new unsigned char[mStride * mFullHeight];

User avatar
kipling
Posts: 91
Joined: Wed Jun 18, 2008 2:36 am

Re: Memory issues

Post by kipling »

OK - this seems a lot more stable. Thanks.

Obviously writing to a canvas bigger than has been allocated is not wise.

I can increase the number of shapes up to 5*10^6 without stability issues. This is in version 3.0.5 both the .App and the CLI.

The memory usage (& swapping) still seems to be a problem in the .App, but I note from the release notes that you are still chasing a memory leak, and it seems to work better after restart.

User avatar
MtnViewJohn
Site Admin
Posts: 882
Joined: Fri May 06, 2005 2:26 pm
Location: Mountain View, California
Contact:

Re: Memory issues

Post by MtnViewJohn »

The memory leak only affects designs that have paths with parameters. Your crystal design doesn't leak and never has more than about 70 parameter blocks allocated at any time. I have successfully run it with 2000x3600 resolution and it produced 70 million shapes and it didn't seem to swap. But I have 8gig of DRAM.

User avatar
MtnViewJohn
Site Admin
Posts: 882
Joined: Fri May 06, 2005 2:26 pm
Location: Mountain View, California
Contact:

Re: Memory issues

Post by MtnViewJohn »

I have noticed another bug with tiled designs that have FILL shapes, like crystals-2013.cfdg, that can cause hangs and memory problems. If you see this then try changing line 1099 in renderimpl.cpp from

Code: Select all

    if (tiler) {
to

Code: Select all

    if (tiler && s.mShapeType != primShape::fillType) {

Post Reply