"JavaScript heap out of memory" when building on RHEL-7

[mcepl]

• VSCode Version: 1.10.2
• OS Version: RHEL-7 (7.3 with nightly packages on the way towards 7.4).
• NodeJS: v6.9.4 (packaged in the EPEL-7 repository)
• NPM: 3.10.10

Steps to Reproduce:

1. Try to build RPM package using https://gitlab.com/mcepl/vscode-pkg/blob/master/vscode.spec
2. Observe the failure

The build log ends with:

<--- Last few GCs --->
  208718 ms: Mark-sweep 1158.0 (1436.2) -> 1158.0 (1436.2) MB, 1781.4 / 0.0 ms [allocation failure] [scavenge might not succeed].
  210553 ms: Mark-sweep 1158.0 (1436.2) -> 1158.0 (1436.2) MB, 1834.4 / 0.0 ms [allocation failure] [scavenge might not succeed].
  212438 ms: Mark-sweep 1158.1 (1436.2) -> 1158.0 (1436.2) MB, 1885.3 / 0.0 ms [allocation failure] [scavenge might not succeed].
<--- JS stacktrace --->
Cannot get stack trace in GC.
FATAL ERROR: MarkCompactCollector: semi-space copy, fallback in old gen Allocation failed - JavaScript heap out of memory
 1: node::Abort() [gulp]
 2: 0x109b1dc [gulp]
 3: v8::Utils::ReportApiFailure(char const*, char const*) [gulp]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [gulp]
 5: v8::internal::MarkCompactCollector::EvacuateNewSpaceVisitor::Visit(v8::internal::HeapObject*) [gulp]
 6: v8::internal::MarkCompactCollector::VisitLiveObjects(v8::internal::MemoryChunk*, v8::internal::MarkCompactCollector::HeapObjectVisitor*, v8::internal::MarkCompactCollector::IterationMode) [gulp]
 7: v8::internal::PageParallelJob<v8::internal::EvacuationJobTraits>::Task::RunInternal() [gulp]
 8: v8::internal::MarkCompactCollector::EvacuatePagesInParallel() [gulp]
 9: v8::internal::MarkCompactCollector::EvacuateNewSpaceAndCandidates() [gulp]
10: v8::internal::MarkCompactCollector::CollectGarbage() [gulp]
11: v8::internal::Heap::MarkCompact() [gulp]
12: v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector, v8::GCCallbackFlags) [gulp]
13: v8::internal::Heap::CollectGarbage(v8::internal::GarbageCollector, char const*, char const*, v8::GCCallbackFlags) [gulp]
14: v8::internal::Factory::NewFillerObject(int, bool, v8::internal::AllocationSpace) [gulp]
15: v8::internal::Runtime_AllocateInTargetSpace(int, v8::internal::Object**, v8::internal::Isolate*) [gulp]
16: 0x1a493cd079a7

The local computer has 4GB free memory (12GB total), 3+GB free disk space on /home partition.

[mcepl]

On local system I get this instead:

<--- Last few GCs --->

  156645 ms: Mark-sweep 1164.1 (1435.0) -> 1163.3 (1435.0) MB, 1243.1 / 0.0 ms [allocation failure] [GC in old space requested].
  157909 ms: Mark-sweep 1163.3 (1435.0) -> 1162.8 (1435.0) MB, 1263.3 / 0.0 ms [allocation failure] [GC in old space requested].
  159185 ms: Mark-sweep 1162.8 (1435.0) -> 1184.0 (1409.0) MB, 1276.2 / 0.0 ms [last resort gc].
  160489 ms: Mark-sweep 1184.0 (1409.0) -> 1205.6 (1409.0) MB, 1303.3 / 0.0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3e87a37cfb39 <JS Object>
    1: hasOwnProperty(this=0x377d6686b409 <an Object with map 0x2898ded30d59>,0x2f695c36ec69 <String[123]: file:///home/matej/build/EXTRAS/vscode/vscode-1.10.2/src/vs/platform/extensionManagement/node/extensionManagementService.ts>)
    2: addMapping(aka SourceMapGenerator_addMapping) [/home/matej/build/EXTRAS/vscode/vscode-1.10.2/node_modules/concat-with-sourcemaps/node_modules/source-map/lib/...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [gulp]
 2: 0x109b1dc [gulp]
 3: v8::Utils::ReportApiFailure(char const*, char const*) [gulp]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [gulp]
 5: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [gulp]
 6: v8::internal::String::SlowFlatten(v8::internal::Handle<v8::internal::ConsString>, v8::internal::PretenureFlag) [gulp]
 7: v8::internal::StringTable::LookupString(v8::internal::Isolate*, v8::internal::Handle<v8::internal::String>) [gulp]
 8: v8::internal::LookupIterator::LookupIterator(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Name>, v8::internal::Handle<v8::internal::JSReceiver>, v8::internal::LookupIterator::Configuration) [gulp]
 9: v8::internal::Runtime_ObjectHasOwnProperty(int, v8::internal::Object**, v8::internal::Isolate*) [gulp]
10: 0x27567bb092a7

[alexdima]

Our gulp tasks are hitting v8 heap limits. Have you tried npm run compile aka gulp compile --max_old_space_size=4096 (https://github.com/Microsoft/vscode/blob/master/package.json#L15)

[mcepl]

Our gulp tasks are hitting v8 heap limits. Have you tried npm run compile aka gulp compile --max_old_space_size=4096 (https://github.com/Microsoft/vscode/blob/master/package.json#L15)

I don’t understand … this has been already set in package.json which is included in your tarball, isn’t it? I am building on top of https://github.com/Microsoft/vscode/archive/1.10.2.tar.gz

[Tyriar]

You can look at our spec file for reference if you want https://github.com/Microsoft/vscode/blob/master/resources/linux/rpm/code.spec.template

But this error is happening as @alexandrudima pointed out because of node's memory limits. You will need to run gulp and add that flag:

node_modules/.bin/gulp vscode-linux-%{arch} --max_old_space_size=4096

[mcepl]

Right, I didn’t got the point that apparently that package.json file is not read and that the command line itself should be modified. When I have done so, build works, even on the remote build system, so it is available now.
Thank you!

[Tyriar]

@mcepl I don't think you can set the --max_old_space_size flag globally to apply to all gulp commands, so we would need to make an explicit npm script for every single gulp task for that to happen. We only hard code it for npm run watch and npm run compile

[mcepl]

don't think you can set the --max_old_space_size flag globally to apply to all gulp commands, so we would need to make an explicit npm script for every single gulp task for that to happen. We only hard code it for npm run watch and npm run compile

No problem, it works now (only on 64bit platforms though, I guess 32bit system just don't have enough RAM, right?), so I don't need to care anymore.

[Tyriar]

Yeah you may not be able to build on 32-bit. You should be able to cross compile a 32-bit version on a 64-bit machine though.

[mcepl]

Yeah, in the end I really don’t care enough. Who in the world would be developing on 32bit machine these days?