[gatsby-transformer-remark] Including length in frontmatter may cause heap out of memory

[c88tm]

Summary

I'm trying to include markdown files using gatsby-transformer-remark following this tutorial, it's great with what the tutorial provided, but once I included my own markdown files which looks like this:

---
title: "TT #12: 沒有關上"
url_link: 12
category: podcast
media: 12.mp3
image: images/itunes.png
date: 2020-06-04 22:37:25
length: 81185414
mediatype: audio/mp3
duration: 5072
---

Content

The website crashes and gives me a FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

After some playing around I figured out that the length property causes this issue, but I can't find any information about why this property is used in an unexpected way. Is there any document I should be aware of ? or should I just have to figure a way to avoid this situation?

Any help would be nice, thanks.

Relevant information

Full error log:

> gatsby develop

success open and validate gatsby-configs - 0.072s
success load plugins - 0.284s
success onPreInit - 0.012s
success initialize cache - 0.012s
success copy gatsby files - 0.121s
success onPreBootstrap - 0.037s
success createSchemaCustomization - 0.018s
⠙ source and transform nodes

<--- Last few GCs --->

[19196:0x2f38810]    12022 ms: Mark-sweep 644.1 (689.6) -> 644.0 (652.1) MB, 326.2 / 0.0 ms  (average mu = 0.543, current mu = 0.000) last resort GC in old space requested
[19196:0x2f38810]    12338 ms: Mark-sweep 644.0 (652.1) -> 644.0 (652.1) MB, 315.8 / 0.0 ms  (average mu = 0.378, current mu = 0.000) last resort GC in old space requested


<--- JS stacktrace --->

==== JS stack trace =========================================

    0: ExitFrame [pc: 0x38a76df5be1d]
Security context: 0x1ebf6449e6c1 <JSObject>
    1: /* anonymous */ [0x26583ab20811] [/home/ubuntu_onino/try_gatsby/hello-world/node_modules/gatsby/node_modules/lodash/lodash.js:~4889] [pc=0x38a76dfd3854](this=0x256ec3b8d461 <JSGlobal Object>,collection=0x22ec712684e1 <Object map = 0x316b322b8b21>,iteratee=0x22ec71268521 <JSFunction _.each (sfi = 0x2cd164ea27e1)>)
    2: forEach [0xff7ddf6e8e1] [/home/...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: 0x8fb090 node::Abort() [node]
 2: 0x8fb0dc  [node]
 3: 0xb033ae v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, bool) [node]
 4: 0xb035e4 v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, bool) [node]
 5: 0xef7642  [node]
 6: 0xf06e5f v8::internal::Heap::AllocateRawWithRetryOrFail(int, v8::internal::AllocationSpace, v8::internal::AllocationAlignment) [node]
 7: 0xecfeb6 v8::internal::Factory::AllocateRawArray(int, v8::internal::PretenureFlag) [node]
 8: 0xed073a v8::internal::Factory::NewFixedArrayWithFiller(v8::internal::Heap::RootListIndex, int, v8::internal::Object*, v8::internal::PretenureFlag) [node]
 9: 0xed0cf7 v8::internal::Factory::NewUninitializedFixedArray(int, v8::internal::PretenureFlag) [node]
10: 0xe92470  [node]
11: 0xe9272f  [node]
12: 0x10094de v8::internal::JSObject::AddDataElement(v8::internal::Handle<v8::internal::JSObject>, unsigned int, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::ShouldThrow) [node]
13: 0x1022c42 v8::internal::Object::AddDataProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::PropertyAttributes, v8::internal::ShouldThrow, v8::internal::Object::StoreFromKeyed) [node]
14: 0x103d75d v8::internal::Object::SetProperty(v8::internal::LookupIterator*, v8::internal::Handle<v8::internal::Object>, v8::internal::LanguageMode, v8::internal::Object::StoreFromKeyed) [node]
15: 0x1198855 v8::internal::Runtime::SetObjectProperty(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::LanguageMode) [node]
16: 0x11999b0 v8::internal::Runtime_SetProperty(int, v8::internal::Object**, v8::internal::Isolate*) [node]
17: 0x38a76df5be1d

Environment (if relevant)

System:
OS: Linux 4.19 Ubuntu 18.04.4 LTS (Bionic Beaver)
CPU: (4) x64 Intel(R) Core(TM) i5-6300U CPU @ 2.40GHz
Shell: 4.4.20 - /bin/bash
Binaries:
Node: 10.22.0 - ~/.local/nvm/versions/node/v10.22.0/bin/node
Yarn: 1.7.0 - /mnt/c/Program Files (x86)/Yarn/bin/yarn
npm: 6.14.6 - ~/.local/nvm/versions/node/v10.22.0/bin/npm
Languages:
Python: 2.7.17 - /usr/bin/python
npmPackages:
gatsby: ^2.23.12 => 2.23.12
gatsby-plugin-sass: ^2.3.12 => 2.3.12
gatsby-source-filesystem: ^2.3.24 => 2.3.24
gatsby-transformer-remark: ^2.8.28 => 2.8.28
npmGlobalPackages:
gatsby-cli: 2.12.68

[pieh]

Hi @c88tm! Thanks for the report, I just reproduced it.

You correctly pointed out length attribute. This is not by design and is bug in the implementation. Exact place in our code is

gatsby/packages/gatsby/src/db/sanitize-node.js

Line 18 in b164147

_.each(data, (o, key) => {

and data in this case will be frontmatter object. The problem here is _.each handling of objects that contain length key ( lodash/lodash#2352 ). I think it makes lodash library think that the object has over 81 million fields :yikes:

I'm not sure if this is only place where we call _.each (or other similar functions) from lodash on the frontmatter object. But it's the first one we hit at least.

I can only offer workaround for now - to rename the field, until proper fix is done

[JAANVI999]

can this issue be assigned to me ?

[Shivam-AfA]

Hi, is someone working on this issue or can it be assigned to me?