Not sure if it will work, but try:

1) Rendering an undirected graph via graphviz's neato tool.

2) Rendering to svg, which might let graphviz not have to keep as much rendering information in memory at any one time.

3) Asking about it on the grapviz-interest mailing list. https://mailman.research.att.com/mailman/listinfo/graphviz-i...