VGA Text Mode

[utterances-bot]

This is a general purpose comment thread for the “VGA Text Mode” post.

[gurpreetshanky]

Excellent Tutorial series.. As a Rust newbie its a great resource to learn about new things.

[davidvedvick]

Wow this was all sorts of fun... thanks for putting this together. I've both learned a lot about Rust (in a far gentler fashion than normal) and learned what's involved in building an OS. I look forward to future articles!

By the way, for whatever reason, when I call my implementation of write_str, I don't have to call .unwrap(), yet I need to call it from the macro... either I did something wrong or there's some difference I have yet to find with the tutorial and my code. Here's my write_str implementation:

impl fmt::Write for Writer {
    fn write_str(&mut self, s: &str) -> fmt::Result {
        for byte in s.bytes() {
            self.write_byte(byte)
        }
        Ok(())
    }
}

[drgmr]

@Danrien I had the same problem - make sure your impl Writer doesn't have a write_str method too - you might be using that instead of the one in the fmt::Write implementation.

[davidvedvick]

Ah thanks, that did the trick!

[phil-opp]

@gurpreetshanky @Danrien Thanks a lot! Glad that you find the posts informative :).

@Danrien @edupc I just pushed #417 to avoid this confusion. Thanks for the feedback!

[SomeAnotherDude]

I think it should be useful to add an empty branch to println! macro. (Standard macro has this branch).

	macro_rules! println {
	    () => (print!("\n"));
	    ($fmt:expr) => (print!(concat!($fmt, "\n")));
	    ($fmt:expr, $($arg:tt)*) => (print!(concat!($fmt, "\n"), $($arg)*));
	}

[fpsiorz]

I don't think printing strings byte by byte is good in this case. The characters on screen are encoded in Code Page 437, which is identical to UTF-8 (which Rust strings use) only for a certain range of characters.

The write_byte currently prints symbols for all control characters except '◙' (which is on the code point of '\n'). I think it should do something like this instead:

pub fn write_byte(&mut self, byte: u8) {
    match byte {
        b'\n' => self.new_line(),
        b' '...b'~' => {
            // keep current behavior
            ...
        },
        _ => {
            // handle non-ASCII characters
        },
    }
}

I added Unicode to Code Page 437 translation to my implementation, but that's a lot of code.

[phil-opp]

@fpsiorz Thanks for the suggestion! I opened #425 to add such a check in the write_strings method.

[arjanvaneersel]

At first I got this error:

error[E0658]: const fn is unstable (see issue #24111)
  --> src/vga_buffer.rs:33:5
   |
33 | /     const fn new(foreground: Color, background: Color) -> ColorCode {
34 | |         ColorCode((background as u8) << 4 | (foreground as u8))
35 | |     }
   | |_____^
   |
   = help: add #![feature(const_fn)] to the crate attributes to enable

The reason is obvious: I forgot to add #![feature(const_fn)] to main.rs, but it took me a while to figure that out and at first I simply removed the "const" at the function declaration. The code executed fine without the const.

So I wonder: What are const functions exactly for and why do we need to use a const function here instead of a normal function? I found the (RFC)[https://github.com/rust-lang/rfcs/blob/master/text/0911-const-fn.md] on const functions, but that doesn't make me understand the concept behind const functions.

[phil-opp]

@arjanvaneersel A const feature behaves exactly the same as a normal function, but has one additional feature: It can be used for initializing const values, such as a static or an array length. In our case, the idea was that it can be used to initialize the static WRITER. However, we later use lazy_static for initializing it, so the const really isn't needed here.

[phil-opp]

@arjanvaneersel I removed the const from the function and added a short explanation of const evaluation in ba266f3.

[bugabinga]

Thank you! This is a lot of fun :).

Typical beginner mistakes, that compile, but lead to blank screen:
(these are all errors I made)

• using vga buffer address 0x8000 instead of 0xb8000 (missing 'b')
• in new_line putting the clear_row call inside the outer for-loop

small "debugging" tip: put some character into the blank.ascii_character field in clear_row to see what is actually happening. I chose `b'_'.

Some notes from a beginner perspective:

• I do not understand what is happening in the unsafe block. At all. Is this by any chance explained somewhere? I am assuming that it is a well know pattern...?
• The unwrap seems suspicious. If it "should" (read: hopefully,maybe) never occur, why is it there? Is there not even one plausible scenario?

Very kool stuff!

If someone else likes to use cargo-make, like I do, here is a usable Makefile.toml:

[tasks.build]
description = "Build no-std rust binary. Combine it with a preconfigured bootloader from rust-osdev."
command = "bootimage"
args = ["build"]

[tasks.boot]
description="Boot the kernel in a qemu virtual machine."
command="bootimage"
args=["run"]

# build 'bootimage.bin' via the bootimage tool
$ cargo make build
# run 'bootimage.bin' in qemu
$ cargo make boot

[phil-opp]

@bugabinga Thanks for your comment!

I do not understand what is happening in the unsafe block.

You mean &mut *(0xb8000 as *mut Buffer)? From the post:

The syntax for this might seem a bit strange: First, we cast the integer 0xb8000 as an mutable raw pointer. Then we convert it to a mutable reference by dereferencing it (through *) and immediately borrowing it again (through &mut). This conversion requires an unsafe block, since the compiler can't guarantee that the raw pointer is valid.

Basically it's a way of transforming an integer to a memory reference. In C, the equivalent would be the cast (Buffer *) 0xb8000.

The unwrap seems suspicious. If it "should" (read: hopefully,maybe) never occur, why is it there? Is there not even one plausible scenario?

The problem is that the fmt::Write trait is defined that way that write_str returns a Result. The reason for this is that some implementations (e.g. for Linux stdout) might return an error. In our implementation, we never return an error, so the unwrap is safe.

[cedws]

Hey, thanks for this excellent blog series. I noticed while following along a small adjustment that could be made. Perhaps it was done like this for clarity, but I think the blog series assumes a familiarity with Rust.

let color_code = self.color_code;
self.buffer.chars[row][col] = ScreenChar {
    ascii_character: byte,
-   color_code: color_code,
+   color_code,
};
self.column_position += 1;

I was also prompted by the latest nightly compiler to add #![feature(exclusive_range_pattern)] due to this chunk of code:

match byte {
    0x20..0x7e | b'\n' => self.write_byte(byte),
    _ => self.write_byte(0xfe),
}

I recommend changing it to look like this, so that 0x7e (which is ~) is included in the pattern. This also means that the feature gate I mentioned above won't be required.

- 0x20..0x7e
+ 0x20..=0x7e

[phil-opp]

@c-edw Thanks, I agree with both suggestions! Could you open pull request that applies these changes to both src/vga_buffer.rs and blog/content/second-edition/posts/03-vga-text-buffer/index.md?

[correabuscar]

Coming from the bottom of this https://os.phil-opp.com/minimal-rust-kernel/ where:

This comment thread directly maps to a discussion on GitHub, so you can also comment there if you prefer.
Instead of authenticating the giscus application, you can also comment directly on GitHub.

either of the above 2 urls are listing both threads:
that is the current one: #999
and the actual one that it should list: #998

So that was confusing until I noticed that one of them is for the next thread ie. https://os.phil-opp.com/vga-text-mode/ , so it must be a "bug".

[phil-opp]

Thanks for reporting and sorry for replying so late! I pushed a fix in #1242, which should avoid the false positives.

[mahmoud-moursy]

I feel like println should not be implemented here; it's better to use writeln! because it makes the locks on the WRITER explicit and helps protect against accidental deadlocks. "Explicit over Implicit" also aligns with the Rust philosophy better imo.

[phil-opp]

Good point! I will consider this for the upcoming third edition!

[Lokathor]

The article says that the VGA text buffer is "generally" 80x25, but is that something that can be configured in some way?

[phil-opp]

(Sorry for the late reply.)

Yes, there are different VGA and VESA modes, depending on the hardware. See https://wiki.osdev.org/VGA_Hardware#Video_Memory_Layout, https://en.wikipedia.org/wiki/VGA_text_mode, and https://en.wikipedia.org/wiki/Video_Graphics_Array#Capabilities . However, changing it is not that easy when using BIOS booting since the BIOS interrupts are only available when the CPU runs in 16-bit real mode.

Because of this, bootloaders typically provide options to configure the VGA mode, which they then apply directly at boot time. Version 0.9 of the bootloader crate doesn't support this unfortunately. The latest version does, but it's not compatible with this tutorial yet. I'm currently working on a new edition of the blog, which will use the new bootloader version to get more interesting screen output.

[ryankav]

Thanks for writing these posts, they are incredibly informative. I just wanted to check if by the end of this article the volatile crate is still needed? When we introduced the dependency the justification was we weren't reading from the buffer so it was added to prevent the compiler optimising it away. However, that was before the implementation of new line was written which does read from the buffer. Is the new line function enough to mean that volatile isn't needed or is it still required?

[bjorn3]

It is still required. Without the compiler is free to delay, reorder or entirely skip the write for as long as the program itself wouldn't be able to observe it. It can for example directly forward the value to be written to the new_line function without it ever touching memory or it can delay writing until calling new_line. It is also free to write garbage first before writing the real value.

[ryankav]

Awesome thank you for clarifying, and responding so quickly!

[sporting6]

Hello, thank you for this series, it was super informative. I was trying to find a good way to implement something that would move the vga buffer's cursor. I was wondering if you could point me in the right direction.

[bjorn3]

https://wiki.osdev.org/Text_Mode_Cursor#Without_the_BIOS describes which IO ports need to be written to for control over the cursor. https://os.phil-opp.com/testing/#using-the-exit-device has an explanation on how to write to IO ports using rust.

[sporting6]

Ive implemented the thing to move the cursor and used the ports but it will not work. I wont print any thing besides the first character i try to print so i think its getting stuck on the update function. Thanks for responding so quickly.

[bjorn3]

The location where the next character is printed is entirely independent of the cursor presented to the user. With VGA text mode you have a single range of memory representing the entire screen. If you write to the first 16 bits this will show up as the top left character on the screen. If you write to the next 16 bits this will show up as the character to the right, and so on.

[ochudi]

Can't lie, I'm really enjoying this. It's consolidating all that I have learned in the classroom... or computer laboratory in my case.

Thank you!

[aiv2008]

There is a problem with running qemu-system-x86_64 -drive format=raw,file=bootimage.bin on wsl. QEMU runs but doesn't display the "Hello World!" text. My wsl version is Archlinux and it works when i use fedora. And i am willing to know the reason.

[boredcoder411]

Use a WSLg distro, which should be any distro on the Microsoft store (not arch)

I personally use WSL debian, never had any problems

[hyunsooda]

Thank you for the excellent tutorial. I have a safety-related question.

The VGA buffer was initialized within an unsafe block. The Rust compiler seems not guarantee access safety to the variable initialized within the unsafe block, even when the accesses are outside of an unsafe block.

Here is a minimized example:

fn main() {    
    // block1
    {
        let buf1 = unsafe { &mut *(0xb8000 as *mut [[u8; 2]; 2]) };
        let _should_not_be_compiled = &buf1[100][100];
    }

    // block2
    {
        let buf2: [[u8; 2]; 2] = [[1, 2], [3, 4]];
        let _should_not_be_compiled = &buf2[100][100];
    }
}

The compiler correctly raised an issue for "block2" as expected. However, "block1" did not trigger any issues. Am I misunderstanding the safety concept highlighted in the tutorial? I would appreciate any comments on this matter. Thank you.

[zhanlidegui]

Buffer declarations are safe code.
let buf1 = unsafe { &mut *(0xb8000 as *mut Buffer) }; let _should_not_be_compiled = &buf1.chars[100][100];
The compiler can pinpoint errors.
index out of bounds: the len is 25 but the index is 110.

[phil-opp]

This is not related to unsafe as bounds checks happen at runtime, as @zhanlidegui said. In some cases, the compiler is smart enough to recognize that some code will always panic at runtime, then it can throw a compile-time error. But this only works for simple cases.

For example, we can make your safe block2 example compile by adding an additional &mut indirection (it still panics at runtime):

    {
        let buf2: &mut [[u8; 2]; 2] = &mut [[1, 2], [3, 4]];
        let _should_not_be_compiled = &buf2[1][100];
    }

[hyunsooda]

I'm sorry. I confused that correct bounds check is available at compile time in theoretically. Now I got it. Thanks.

[0xnetfox]

This blew my mind away... as someone that has followed the equivalent C VGA implementation documented on OSDev, it's incredible how easy it is to implement it in rust. And it doesn't only work on strings, but on all primitives and stuff that derives Debug, no need to cry implementing all that yourself :')

Great work on these guides, been enjoying them a lot so far!

[AKgit834]

i am getting this error and i have checked my code multiple times ::
error: linking with rust-lld failed: exit status: 1
|
= note: LC_ALL="C" PATH="/home/aadityakaushik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-unknown-linux-gnu/bin:/home/aadityakaushik/.cargo/bin:/home/aadityakaushik/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin" VSLANG="1033" "rust-lld" "-flavor" "gnu" "/tmp/rustcRfe5IM/symbols.o" "/home/aadityakaushik/mine/vs/rust/os/target/specs/debug/deps/os-77a1319f10335eec.125ktcj43x1j77lw.rcgu.o" "/home/aadityakaushik/mine/vs/rust/os/target/specs/debug/deps/os-77a1319f10335eec.4bxv69p99dqq6ztg.rcgu.o" "--as-needed" "-L" "/home/aadityakaushik/mine/vs/rust/os/target/specs/debug/deps" "-L" "/home/aadityakaushik/mine/vs/rust/os/target/debug/deps" "-L" "/home/aadityakaushik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/specs/lib" "-Bstatic" "/home/aadityakaushik/mine/vs/rust/os/target/specs/debug/deps/librustc_std_workspace_core-bd2f432810c8a3c0.rlib" "/home/aadityakaushik/mine/vs/rust/os/target/specs/debug/deps/libcore-06e145513a31b9ac.rlib" "/home/aadityakaushik/mine/vs/rust/os/target/specs/debug/deps/libcompiler_builtins-0b72f5350e608b72.rlib" "-Bdynamic" "--eh-frame-hdr" "-z" "noexecstack" "-L" "/home/aadityakaushik/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/specs/lib" "-o" "/home/aadityakaushik/mine/vs/rust/os/target/specs/debug/deps/os-77a1319f10335eec" "--gc-sections"
= note: rust-lld: error: undefined symbol: memset
>>> referenced by num.rs:217 (src/fmt/num.rs:217)
>>> core-06e145513a31b9ac.core.2d03f973ff1cd8f1-cgu.13.rcgu.o:(core::fmt::num::imp::fmt_u64::h29d6854c72ef82b1) in archive /home/aadityakaushik/mine/vs/rust/os/target/specs/debug/deps/libcore-06e145513a31b9ac.rlib

error: could not compile os (bin "os") due to 1 previous error

help please i have no idea what it is saying.

[AtomicGamer9523]

did you do this:

[unstable] 
 build-std = ["core", "compiler_builtins"] 
 build-std-features = ["compiler-builtins-mem"] 
  
 [build] 
 target = "x86_64-blog_os.json" 
  
 [target.'cfg(target_os = "none")'] 
 runner = "bootimage runner"

[0xOmarA]

Small point, it might be simpler to implement new_line by doing a rotate left over the entire array. I think that this simplifies the implementation quite a bit and makes it clear what we're trying to achieve. Here is an example:

fn new_line(&mut self) {
    self.buffer.rotate_left(1);
    *self.buffer.last_mut().unwrap_unchecked() = [Char::black_whitespace(); BUFFER_WIDTH];
    self.cursor.reset()
}

[phil-opp]

Nice, i didn't know that there is a rotate_left function in the standard library!

[VincentBerthier]

From what I see, rotate_left (and all its derivatives) are in the std library, not sure how you could use that in a no_std environment.
There is something you can do which is almost as good, and go for swaps:

fn new_line(&mut self) {
    (1..BUFFER_HEIGHT).for_each(|row| self.buffer.chars.swap(row - 1, row));
    let blank = ScreenChar::blank(self.color_code);
    self.buffer.chars[BUFFER_HEIGHT - 1] = from_fn(|_| Volatile::new(blank));
    self.column_position = 0;
}

[drzewiec]

A lot of functions (including rotate_left) are in core as well as std. So there's no problem using it in a no_std crate.

[VincentBerthier]

Uh, it didn’t get picked on my LSP for some reason…but now it is… Oh well. All good!

[KikkiZ]

I was using the latest volatile (0.5.1) and bootloader (0.11.4) and the compiler was reporting a bunch of errors, and then I found that the main error was on the bootloader, and I wondered if it was because of a version issue, or if the new version needed to do additional configuration

[bjorn3]

You need bootloader 0.9 for the second edition. Bootloader 0.11 is used by the still being written third edition of blog os.

[KikkiZ]

I see, I just wanted to confirm that this is not my issue, thanks for the reply.

[qvd808]

Thanks for the tutorial. I have been enjoy learning it so far.

[SummerGram]

Hi, it is an excellent tutorial.

I have a question about writing to VGA buffer. I am trying to write the character to the VGA buffer at the position more than BUFFER_WIDTH. The text will not be displayed in the VGA.

I suppose there is checking in runtime but I don't see any panic message in the runtime. Will buffer overflow happen?

    pub fn write_byte(&mut self, byte: u8) {
        match byte {
            b'\n' => self.new_line(),
            byte => {
                let color_code = self.color_code;
                self.buffer.chars[10000][10000].write(ScreenChar {
                    ascii_character: byte,
                    color_code,
                });
            }
        }
    }

Thanks!

[KikkiZ]

The width and height of the VGA are fixed, and I think that for content that exceeds the width and height is not in its display range, it may automatically discard the content beyond without panicking

[SummerGram]

Is it possible to create a byte variable to check that?

[KikkiZ]

I think you can set the size of the buffer to the same size as the VGA (80*25) when you use the vga-related interface, so that you don't need to manually check if it is out of range; However, you may want to keep the context in the buffer, which requires a larger buffer size, which requires some variables to check.

[phil-opp]

I suppose there is checking in runtime but I don't see any panic message in the runtime. Will buffer overflow happen?

Rust always checks for out of bounds access on arrays, so a panic does happen. However, you probably don't see the panic because the WRITER is still locked when your panic function is called, so a deadlock occurs.

You can try to force-unlock the WRITER by calling unsafe { WRITER.force_unlock() }; at the beginning of your panic function. If you do that, you will likely see QEMU rebooting over and over again. The reason for that is that the println! call in your panic handler will run into the out-of-bounds write again, causing a another panic. This continues until your stack overflows and a page fault exception occurs. As we don't handle CPU exceptions yet (we do it later in the "CPU Exceptions" post), a so-called triple-fault occurs, which causes QEMU to reset the system.