Transferring big file

[mauroForlimpopoli]

In my app I created a file of about 125 kbyte.
Unfortunately I haven't memory enough to send it all together.
I can send it on "serial" or "ble" in small package, but is there is a way to upload it in FTP mode ?
Which is the simplest way to read it ?
Regards

[phoddie]

You can do an HTTP POST in pieces. That's probably the easiest approach to getting a file from your device onto a web server.

The httppoststreaming example shows how to make a single POST to a server using multiple fragments. That could be easily adapted to send a file instead.

[mauroForlimpopoli]

Good morning Peter,
I tried some solutions, but I have some again some problems.
This is my code, and the file to upload is about 125 kB

function uploadFile() {
	busyNet = 1
trace("up")
	let	file2 = new File(root + "logger.bin", true);
	file2.position = 0;
	
	let request = new Request({
		host: "httpbin.org",
		path: "/post",
		method: "POST",
		body: true,		// callback is provided by Request.requestFragment callbacks
		headers: ["Content-Length", file2.length, "Content-Type", "text/plain"],
		response: String
	});
	
	//request.bytesToSend = kBodyPart.length * kRepeat;
	request.bytesToSend = file2.length
	let maxBuff
	if (file2.length < 128) {
		maxBuff = file2.length
	} else {
		maxBuff = 128
	}
	
	request.callback = function(message, value) {
		if (Request.requestFragment === message) {
			if (0 === this.bytesToSend)
				return;		// no more body

			let shortsRead = new Uint8Array(file2.read(ArrayBuffer, maxBuff));
			this.bytesToSend -= maxBuff;
			if (this.bytesToSend < 0){
				maxBuff = maxBuff + this.bytesToSend // riduco ultimo buffer es: se buffer = -12  nuovo buffer = 128 - 12
				this.bytesToSend = 0
				busyNet = 0
			}

			return shortsRead;	// fragment of body
		}
		else if (Request.responseComplete === message) {
			busyNet = 0 // 
			value = JSON.parse(value);
		}
	}
}

and it works, but unlucky, in my project, I have about 9 kB of free ram and from the XBug it seems that this function use about 20 kB and my app goes in Memory full. How can I reduce the Ram use?
Regards
Thank you

[phoddie]

This function, by itself, is likely not using 20 KB.

Characterizing available memory requires more than a single number. There is the memory allocated to the JavaScript visual machine. As I recall, you have that set to a large number in your manifest. But, how much space is free in the slot and chunk heaps inside that? There is also system memory, which is separate from the slot and chunk heaps. That is used by native code, for example the Wi-Fi stack and lwip socket library. How much free system memory do you have? When the "out of memory" error occurs, is that in the slot, chunk, or system memory?

Did you enable Wi-Fi for this function or was Wi-Fi already enabled? That matters because just turning on Wi-Fi consumes about 30 KB on the ESP32 with the current ESP-IDF. Each socket is relatively light, however, just a few KB typically.

I believe your project uses a display. If that's true, you might benefit from updating your Moddable SDK. There is a commit from the last week that reduced the memory used by the SPI driver significantly. That will free up additional system memory, though it will not impact the slot or chunk heaps. If either of those is the source of the failure, the increased system memory is unlikely to make a difference.

[mauroForlimpopoli]

I done your update about Moddable SDK and now I have about 40kB of free RAM more and this is great !
By looking from Xbug I soppose the Memory full is caused from the Slot user Memory. Normally I use about 33000 kB of 50000 kB available.
WiFi is enable just in the start of the project. Now I'm looking how to reduce the use of memory slot during file transfer. Have you some suggestions ?
Regards

[phoddie]

Now I'm looking how to reduce the use of memory slot during file transfer

The memory used by the file transfer is unlikely to be the issue. It is well optimized already.

The memory used by your project before starting the file transfer is more likely the issue.

A good starting point for reducing memory use inside the JavaScript virtual machine is to ensure that as many modules as possible are preloaded. If done properly, that can move the slots used by functions and classes from RAM into flash storage.

[mauroForlimpopoli]

In the xbug all the modules are blue, so I think that are preloaded.
I'm trying to preload main and functions but I have some difficulties.

[mauroForlimpopoli]

If I understood right, the functions cann't be freeze directly, but is the variable inside it that must be freeze.
es:

function decodSens(val) {
	if(val == 252) {
		val = "ON"
	} else if (val == 251) {
		val = "Off"
	} else if (val == 1) {
		val = "Out"
	} else {
		val = val + 1
		val = val / 2
		val = val - 60
	}
	return val	
}

can be freeze in something like this

const valDecodeSens = [];
for (let i = 2; i <= 250; i++)
	valDecodeSens[i] = i - 60
valDecodeSens[1] = "Out"
valDecodeSens[251] = "Off"
valDecodeSens[252] = "On"
Object.freeze(valDecodeSens);

function decodSens(x) {
	if(x == 252 || x == 251 || x == 1) {
		x = x
	}  else {
		x = x + 1
		x = x / 2
		x = x - 60
	}
	return valDecodeSens[x];
}

or these is a better way ?

[phoddie]

You have misunderstood.