Re: [COFF] [TUHS] more about Brian... [really Rust]

[TUHS to Bcc, +COFF &lt;coff(a)minnie.tuhs.org&gt; ] This isn't exactly COFF material, but I don't know what list is more appropriate. On Thu, Feb 3, 2022 at 9:41 PM Jon Steinhart &lt;jon(a)fourwinds.com&gt; wrote: I'm a Rust programmer, mostly using it for bare-metal kernel programming (though in my current gig, I find myself mostly in Rust userspace...ironically, it's back to C for the kernel). That said, I'm not a fan-boy for the language: it's not perfect. I've written basically four kernels in Rust now, to varying degrees of complexity from, "turn the computer on, spit hello-world out of the UART, and halt" to most of a v6 clone (which I really need to get around to finishing) to two rather more complex ones. I've done one ersatz kernel in C, and worked on a bunch more in C over the years. Between the two languages, I'd pick Rust over C for similar projects. Why? Because it really doesn't just do the same old things: it adds new stuff. Honest! Further, the sad reality (and the tie-in with TUHS/COFF) is that modern C has strayed far from its roots as a vehicle for systems programming, in particular, for implementing operating system kernels ( https://arxiv.org/pdf/2201.07845.pdf). C _implementations_ target the abstract machine defined in the C standard, not hardware, and they use "undefined behavior" as an excuse to make aggressive optimizations that change the semantics of one's program in such a way that some of the tricks you really do have to do when implementing an OS are just not easily done. For example, consider this code: uint16_t mul(uint16_t a, uint16_t b) { return a * b; } Does that code ever exhibit undefined behavior? The answer is that "it depends, but on most platforms, yes." Why? Because most often uint16_t is a typedef for `unsigned short int`, and because `short int` is of lesser "rank" than `int` and usually not as wide, the "usual arithmetic conversions" will apply before the multiplication. This means that the unsigned shorts will be converted to (signed) int. But on many platforms `int` will be a 32-bit integer (even 64-bit platforms!). However, the range of an unsigned 16-bit integer is such that the product of two uint16_t's can include values whose product is larger than whatever is representable in a signed 32-bit int, leading to overflow, and signed integer overflow is undefined overflow is undefined behavior. But does that _matter_ in practice? Potentially: since signed int overflow is UB, the compiler can decide it would never happen. And so if the compiler decides, for whatever reason, that (say) a saturating multiplication is the best way to implement that multiplication, then that simple single-expression function will yield results that (I'm pretty sure...) the programmer did not anticipate for some subset of inputs. How do you fix this? uint16_t mul(uint16_t a, uint16_t b) { unsigned int aa = a, bb = b; return aa * bb; } That may sound very hypothetical, but similar things have shown up in the wild: https://people.csail.mit.edu/nickolai/papers/wang-undef-2012-08-21.pdf In practice, this one is unlikely. But it's not impossible: the compiler would be right, the programmer would be wrong. One thing I've realized about C is that successive generations of compilers have tightened the noose on UB so that code that has worked for *years* all of a sudden breaks one day. There be dragons in our code. After being bit one too many times by such issues in C I decided to investigate alternatives. The choices at the time were either Rust or Go: for the latter, one gets a nice, relatively simple language, but a big complex runtime. For the former, you get a big, ugly language, but a minimal runtime akin to C: to get it going, you really don't have to do much more than set up a stack and join to a function. While people have built systems running Go at the kernel level ( https://pdos.csail.mit.edu/papers/biscuit.pdf), that seemed like a pretty heavy lift. On the other hand, if Rust could deliver on a quarter of the promises it made, I'd be ahead of the game. That was sometime in the latter half of 2018 and since then I've generally been pleasantly surprised at how much it really does deliver. For the above example, integer overflow is defined to trap. If you want wrapping (or saturating!) semantics, you request those explicitly: fn mul(a: u16, b: u16) -> u16 { a.wrapping_mul(b) } This is perfectly well-defined, and guaranteed to work pretty much forever. But, my real issue came from some of the tutorials that I perused. Rust is That's one way to look at it, but I don't think that's the intent: the model is rather, "immutable by default." Rust forces you to think about mutability, ownership, and the semantics of taking references, because the compiler enforces invariants on all of those things in a way that pretty much no other language does. It is opinionated, and not shy about sharing those opinions. To me, this implies a mindset that programming errors are more important It's funny, I've had the exact opposite experience. I have found that it actually forces you to invest a _lot_ more in-up front thought about what you're doing. Writing code first, and then sprinkling in `mut` and `unsafe` until it compiles is a symptom of writing what we called "crust" on my last project at Google: that is, "C in Rust syntax." When I convinced our team to switch from C(++) to Rust, but none of us were really particularly adept at the language, and all hit similar walls of frustration; at one point, an engineer quipped, "this language has a near-vertical learning curve." And it's true that we took a multi-week productivity hit, but once we reached a certain level of familiarity, something equally curious happened: our debugging load went way, _way_ down and we started moving much faster. It turned out it was harder to get a Rust program to build at first, particularly with the bad habits we'd built up over decades of whatever languages we came from, but once it did those programs very often ran correctly the first time. You had to think _really hard_ about what data structures to use, their ownership semantics, their visibility, locking, etc. A lot of us had to absorb an emotional gut punch when the compiler showed us things that we _knew_ were correct were, in fact, not correct. But once code compiled, it tended not to have the kinds of errors that were insta-panics or triple faults (or worse, silent corruption you only noticed a million instructions later): no dangling pointers, no use-after-free bugs, no data races, no integer overflow, no out-of-bounds array references, etc. Simply put, the language _forced_ a level of discipline on us that even veteran C programmers didn't have. It also let us program at a moderately higher level of abstraction; off-by-one errors were gone because we had things like iterators. ADTs and a "Maybe" monad (the `Result<T,E>` type) greatly improved our error handling. `match` statements have to be exhaustive so you can't add a variant to an enum and forget to update code to account in just that one place (the compiler squawks at you). It's a small point, but the `?` operator removed a lot of tedious boilerplate from our code, making things clearer without sacrificing robust failure handling. Tuples for multiple return values instead of using pointers for output arguments (that have to be manually checked for validity!) are really useful. Pattern matching and destructuring in a fast systems language? Good to go. In contrast, I ran into a "bug" of sorts with KVM due to code I wrote that manifested itself as an "x86 emulation error" when it was anything but: I was turning on paging very early in boot, and I had manually set up an identity mapping for the low 4GiB of address space for the jump from 32-bit to 64-bit mode. I used gigabyte pages since it was easy, and I figured it would be supported, but I foolishly didn't check the CPU features when running this under virtualization for testing and got that weird KVM error. What was going on? It turned out KVM in this case didn't support gig pages, but the hardware did; the software worked just fine until the first time the kernel went to do IO. Then, when the hypervisor went to fetch the instruction bytes to emulate the IO instruction, it saw the gig-sized pages and errored. Since the incompatibility was manifest deep in the bowels of the instruction emulation code, that was the error that returned, even though it had nothing to do with instruction emulation. It would have been nice to plumb through some kind of meaningful error message, but in C that's annoying at best. In Rust, it's trivial. https://lexi-lambda.github.io/blog/2019/11/05/parse-don-t-validate/ 70% of CVEs out of Microsoft over the last 15 years have been memory safety issues, and while we may poo-poo MSFT, they've got some really great engineers and let's be honest: Unix and Linux aren't that much better in this department. Our best and brightest C programmers continue to turn out highly buggy programs despite 50 years of experience. But it's not perfect. The allocator interface was a pain (it's defined to panic on allocation failure; I'm cool with a NULL return), though work is ongoing in this area. There's no ergonomic way to initialize an object 'in-place' (https://mcyoung.xyz/2021/04/26/move-ctors/), and there's no great way to say, essentially, "this points at RAM; even though I haven't initialized it, just trust me don't poison it" ( https://users.rust-lang.org/t/is-it-possible-to-read-uninitialized-memory-w… -- we really need a `freeze` operation). However, right now? I think it sits at a local maxima for systems languages targeting bare-metal. - Dan C.