Tracking Issue for Second Edition

[phil-opp]

Create a second edition of the blog, which reorders the posts (exceptions before page tables) and uses an own bootloader. This should make the posts easier to follow.

---

Edit: This issue will serve as a tracking issue. The “Second Edition” milestone contains all associated issues and pull requests.

Planned high-level changes

• Create a custom, dependency free bootloader
  • Written completely in inline assembly to be dependency free (see this comment for details)
  • First prototype: https://github.com/rust-osdev/bootloader
• Work out a new order of posts
  • Remove the sudden jump in difficulty with the “Page Tables” post
  • My current plan: Setup & printing first, then exceptions and hardware interrupts (including keyboard and timer support, and maybe even multithreading), then memory management
• Move more code into the x86_64 crate
  • Remove low-level implementation details (e.g. how to align a memory address) from the blog text, but …
  • Document the code in the x86_64 crate very well and ecourage readers to look at it

Further changes

• Link using LLD (Link using lld #370)
  • LLD is about to be shipped with rustc: rust: Import LLD for linking wasm objects rust-lang/rust#48125
• “Kernel Heap”: include more allocator types
• [ ] Put the kernel in the higher half of the address space (see Tracking Issue for Second Edition #360 (comment)) maybe sometimes in the future when we introduce linker scripts
• Unit tests and other tests? (see Add testing framework for the kernel robert-w-gries/rxinu#32 for an overview of possibilities)
• Be platform independent: The blog should work on Linux, Mac, and Windows
  • There are CI jobs that verify this.

Other planned things

• Use rustfmt-nightly so that the code always has standard format. Check formatting via travis.
• Docker support (Add docker support #369)
• Make sure that every code snippet has a comment that specifies the file path (Make sure that every code snippet has a comment that specifies the file path #382).

Maybe

• [ ] Use a javascript emulator in blog posts (Use javascript emulator in blog posts #368) Postponed, since there is no 64-bit js emulator yet

---

(This list is far from complete and will be extended over time.)

[robert-w-gries]

Do you have any plans to re-implement the kernel code in 100% Rust? I noticed that redox manages to use only Rust for kernel code and thought that might be a good goal for the second edition.

Note that there are a dozen calls to the asm! macro in their kernel.

And thank you for the work you've put into the blog! It's been so helpful for both getting me started with a Rust kernel and teaching OSDev in general!

[phil-opp]

Thanks, glad to hear!

Do you have any plans to re-implement the kernel code in 100% Rust?

Yes, I even plan to write the bootloader in inline assembly, so that you don't need any other tools to build a bootable disk image (CD images are harder, but maybe we will support that too). The only non-Rust tools we will need are a linker (though I hope that the LLVM linker lld is shipped with the Rust compiler soon) and binutils (stuff like objdump or gdb; we only need it for debugging).

[donald-pinckney]

This would be super awesome, I literally can't wait for it.

[ketsuban]

Do you have any plans to put the kernel in the higher half? It's not too hard to do (albeit very easy to get wrong), it's something that isn't terribly well-documented for 64-bit, and it keeps you honest about physical and virtual addresses.

[IsaacWoods]

I also think a higher-half design would teach a lot more. I've come across a few bugs in the tutorial code from where physical and virtual addresses got sort of mixed up (due to the identity mapping, these weren't picked up on I'm guessing) and it would set us up nicely for user-land programming!

[phil-opp]

That's a good idea, thanks for the suggestion. I added it to the list.

[robert-w-gries]

I looked into #368 more and determined that it it will be a long time before it can be implemented for blog_os. It should be removed from the Second Edition milestone.

The issue is simply that 64-bit x86 javascript emulators do not exist yet. The most mature js emulator I could find is v86, which only supports 32 bit. The maintainer has no immediate plans to implement 64 bit, but the maintainer is working on a WebAssembly port that will make it possible to support 64 bit mode later.

I still believe that a javascript emulator has value, but it's clear that it won't be relevant to this project for a while. Maybe I'll write one in Rust so we can implement it for blog_os's Tenth Edition 😉

[ketsuban]

One other thing: the blog currently recommends --gc-sections to deal with missing symbols, but I feel like that's kicking the can down the road rather than solving the problem. The solution ought to be to include compiler-builtins, but for some reason it's missing __floatdisf and __floatundisf.

It appears to be simple enough to add them (one macro invocation each) but there may be a good reason for not doing that which I don't know because I'm way out of my depth; in the event it's just that simple, I have a patch ready to go.

[phil-opp]

@robert-w-gries Thanks for the update! The lack of 64-bit support is a pity. So I guess we have to wait for the web assembly port.

@ketsuban I think you're right that including compiler_builtins is the better solution, I added it to the list. I don't know about the __floatdisf symbols, so let's see how they react to your issue. It seems like it isn't getting much attention, so maybe ping someone or ask on IRC after the holidays.