Well, it has been almost 2 weeks since my last post on this thread. Since there is so little information about LSX online, I might as well post all that I have done / noticed.
First things first, the kernel building issue was as simple as I was expecting. All of the build scripts are meant for an external V6 system, and do not work on LSX itself. I forgot that the LSX linker defaults to 040000 instead of 0 which obviously broke my kernel. Setting the "-a" flag on the linker and fixing config stuff in param.h, header.s, and mch.s was all that was needed to make it work.
Next up was converting the userspace from 16K to 20K. I tried to use an external compiler as little as possible and was mostly successful doing so. "ed.c" needed to be broken up into 2 different files, but everything else worked. The only thing that needed to be cross compiled was the C compiler itself. As it turns out, the stock C compiler that LSX comes with does not have enough "OSSIZ" in order to build the 2nd pass of the compiler. Interestingly enough, compiling it with the full space only makes the 2nd pass go up to 23K-ish size. It just barely fits in the userspace, but it does work. I don't know why the original creators of the root image didn't start with this.
Speaking of the C compiler, mounting the "cc.dsk" file from the archives on LSX is a bad idea. Unlike every other image, it is formatted for 500 blocks instead of 400. Trying to write to this filesystem will cause the swap space to get overwritten, which is generally not a good thing.
After the kernel and userspace were working, I went ahead and started making modifications to the kernel. The first goal involved re-adding the RAW tty mode. Turns out, this was super simple and only took like 10 minutes of copying "if" statements. After that, my custom V6 screen editor compiled and worked flawlessly.
Finally, tonight I was able to get a RK05 driver to work alongside the default RX01 driver. This one was a little bit more of a challenge, as all of the block device switch code has been ripped out of LSX. Device drivers also work slightly differently, as some driver support functions are removed and "buf.h" is set up to only use 8-bit dev IDs. After adding back a simplified "bdevsw[]", some modifications to "bio.c", and a whole lot of tinkering I was able to get RK0 auto-mounted on "/mnt". Should make moving lots of files *much* easier, plus it will facilitate my future plans for the project.
I think my next goal is to add back the "mount" and "umount" syscalls. I got about 2200 bytes free of kernel space, so that should be more than enough room to add those functions in. After that, i'll just need to write a RX02 driver and make the jump over to real hardware. Of course, the mystery swap bug still persists.
Thanks for reading,
Gavin