Process Management

[noxer]

Implement a process list and functions for loading, unloading and switching processes.

[rustyoz]

What kind of ABI are we going to use? Same as AtlasOS?

[aponigricon]

I think we should abandon concepts such as an "assembler relocation table", because if the relocation table doesn't list a pointer which the program writes to, the entire system will fail over time. And also, the OS would have to scan through the program code before executing it to change the pointers to point to the virtual memory anyways, so can't it just detect when a program needs memory automatically? The only useful purpose of a relocation table on a 16-bit system would be to keep track of segments, but AtlasOS/2 doesn't use segments?

[noxer]

@bungao: maybe more modular. one function for loading the process into a page, one for relocating it, one for creating the process list entry. system calls via interrupt, device access via file system.

@VladVP: why not both. the automatic relocation has its restrictions. if you search the code for SET PC, 0xnnnn you may relocate data (when we are missing a data segment). A automatic relocation may work for short programs but not for more complex applications. Modern OSs still use relocation tables for libraries and ASR.

Maybe we should use some kind of file header defining the application as self relocating, table relocatable or automatically relocatable.

[aponigricon]

@noxer
Uhh... I don't get why automatic relocation has any restrictions at all 😕

[rustyoz]

To me automatically scanning programs seems more likely to cause errors than a relocation table. at least a relocation table is pretty easy to create and use.

[noxer]

@VladVP:
SET A, data
SET B, buffer
SET C, 10
JSR mem_copy
SUB PC, 1

:data
dat 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
:buffer
dat 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0

Tell me how you find out you should relocate SET A, data and SET B, buffer but not SET C, 10...

[aponigricon]

@bungao
It can both be a charm and a curse... As said before, the entire system breaks if some pointers aren't listed in the relocation table... If we HAVE to use relocation tables, couldn't we at least implement some kind of confirmation system in the loader that confirms that pointers are in the relocation table? That would also make sure that we won't load any unused pointers into VMEM...
Without this kind of implementation, filthy HA crackers could just create an atlas-executable which is designed to clear out memory; listing a few to none memory pointers in the relocation table...

[aponigricon]

@noxer
Very very simple...
If the program _writes_ to an address, then the address must be a pointer no matter what! If it just simply uses a number, but without writing anything to it, then we shoulden't care at all!

[noxer]

@VladVP: 1. Protip: The filthy HA hackes could simply enable interrupt queueing and wait for the DCPU to catch fire.
We would have to simulate a DCPU16 on a DCPU16. Another possibility would be support for some kind of bytecode that is checked for integrity on JITing.

@VladVP: 2. But the actual writing code may be part of the API and may run in the kernel or in a library. And what about self-modifing code? There is always a way around the relocation. The most secure way would be some kind of AtlasBASIC that is compiled on the DCPU where every memory access is checked.

[aponigricon]

@noxer

That is a completely unrelated security issue... what do the interrupt queuing vulnerability have to do with process management? If it's such a big problem, can't we just replace all IAS's with something that modifies something inside the actual interrupt handler which makes it jump to the place in VMEM which the program expects to be the interrupt handler?

If the writing code is in the kernel or API, then the loader will know it, and if it's a part of a library, then what prevents the loader from scanning a function in the library? But self-modifying code sounds like a huge problem... how intelligent is the loader allowed to be?

[rustyoz]

or we could use a relocation table since it works

[aponigricon]

not always...

[rustyoz]

example?

[noxer]

@VladVP: My point was, that you can never ensure integrity when using pointers. We are unable to determine what is code and what is data (or what is data and will be code soon). We can make checks on OS functions if the addresses provided are valid but direct access will always be possible (we need some kind of memory management unit).
There are two ways around it. 1st: connect two DCPUs with one running the OS and one running the application. 2nd: Some kind of interpreted, high-level code that allows us to control the application.
The relocation of code has nothing to do with security, even the automatic relocation can fail and the application may overwrite the interrupt handler.
The security concept of Atlas2 should be on a higher level (some kind of interpreted code).

My suggestion is to make the relocation modular. When parsing the file header the parser calls the matching relocation routine. Since most of the assemblers are able to generate a relocation table and it is the easiest approach we can implement it first. The automatich relocation includes the decoding of the DCPU bytecode which is more complicated. I would suggest coding this in C.

[aponigricon]

@noxer: Well... then I guess we can have both...

The automatich relocation includes the decoding of the DCPU bytecode which is more complicated. I would suggest coding this in C.

I don't understand why you think it's that big of a deal... I mean all info we need about the DCPU-16 machine code is right here:
http://dcpu.com/dcpu-16/
http://0x10cwiki.com/wiki/Instruction_set
And the 0x10cwiki pages for each individual instructions...
Using all of the above data, it should be fairly easy to mash together a machine code decoder... even in pure assembly!!

[rustyoz]

that still doesn't get away from the fact that set a, label will be no different from set a, 0xabcd in the byte code. You would need to have a way to distinguish between the 2. A machine code decoder is just a disassembler, and unless you have debugging information as to what values are pointers and what values are just plain values, relocating code will be difficult.

[aponigricon]

But it doesn't matter as long as the program doesn't use it as a pointer!
For example in SET PC, 0xCAFE, 0xCAFE will always be a pointer no matter what. And in SET [0xDEAD], [0xBEEF], 0xDEAD and 0xBEEF must always be pointers no matter what also. But for example in SET A, 0X4D, we don't know whether 0X4D is a pointer or not, but it doesn't matter either! Even if we knew that it was a pointer, what could we possibly use that information to?

[rustyoz]

that is my point. in your example SET A, 0x4d could be or not be a pointer to a data location within the program. If it is when you relocate the code you will have to offset 0x4d by some value. On the other hand 0x4d could be just a value used in some arithmatic or logic.
For example

SET A, label
ADD A, 2
SET B, 0xcafe
SET [A], B

would set the memory location at label+2 to 0xcafe. now what if when compiled the memory location of 'label' was 0xcafe. The byte code for "set a, label" and "set b, 0xcafe" would be nearly identical 7C01 CAFE versus 7C21 CAFE.
If you envisage scanning the bytecode and changing the literal in SET 'register', 'literal' instructions then it would mess up the code.

Even if we knew that it was a pointer, what could we possibly use that information to?

Relocate the code?

[aponigricon]

Wikipedia just explained to me how kernels like Linux and Windows NT handle self-modifying code. It's actually mighty simple 😆

Oh, well... but now that you've made me realize this issue... uhm... the part about relocation is probably the only issue... Offsetting of pointers can easily be detected by most smart program loaders.
Well... uhmm,.. .... .... thinking.... ... there aren't really any too good robust solutions, are there? 😞
At least we can ensure that programs don't access bad memory with the tecniques that we talked about above... then I guess I don't have any good arguments against relocation tables anymore... okay, you win. 😒

[rustyoz]

it not about winning and loosing. just what works. If you come up with a way of automatically relocating code that works then it would make sense to include it.

[aponigricon]

It was meant with a rather comedic sense in mind 😉