Show Posts
1
Firmware / Migrating the build environment/true portability
« on: February 04, 2026, 12:04:30 pm »
Hi All,
I wanted to again express my appreciation to Raptor CS for access to the hardware. I used that access to do a lot of bare metal testing as well as access to the lowest levels of the firmware, in an effort to build a truly portable build environment. This has been a fantastic learning experience. I sent the following summary to Raptor a few months ago, but then took some time off from computers and failed to post the summary to the forums (unless my memory is worse than I remember).
I have done a lot of digging into the internal boot process, specifically the SBE, and while in theory the boot process can be replaced, in practice this appears to be extremely labor intensive. I have been working with Public Domain POSIX Make
https://frippery.org/make/
to rewrite the Makefiles to be strict POSIX 2024. The GNU syntax in the Makefiles ties the build process very closely to GNU toolchain. They are not compatible with FreeBSD's make. The solution is strict POSIX 2024, but there are 28 Makefiles and 30 .mk files that need to be rewritten.
Note that strict POSIX 2024 does not support out-of-tree building, where the source code is in one directory and the object files are in another. The main cause (effect?) of this is that implicit target rules can not use variables, so $(OBJDIR).o: is not a valid target. The IBM writers of the SBE Makefiles leaned very heavily on GNU's extensions, to the point where I would describe the effort as "lazy." I suspect at least a few of the Makefiles were generated by scripts.
I have rewritten several Makefiles, enough to dig deeper into the SBE build process. The C++ implementation/includes on FreeBSD are not compatible with Linux's C++ implementation. The naming for #ifndef is different, leading to duplication of std functions. In a particular case, xip, the two source files are .C, but one of them does not contain any C++ while the other only contains five lines total, out of 3,237 in the file. That file fails to compile because of the duplicate:
In file included from p9_xip_tool.C:41:
In file included from /usr/include/c++/v1/string:591:
In file included from /usr/include/c++/v1/__algorithm/remove.h:12:
In file included from /usr/include/c++/v1/__algorithm/find.h:16:
In file included from /usr/include/c++/v1/__bit/countr.h:15:
In file included from /usr/include/c++/v1/__bit/rotate.h:15:
In file included from /usr/include/c++/v1/limits:581:
/home/tkelly/raptor/talos-sbe/src/import/chips/p9/procedures/ppe/pk/../include/std/type_traits:46:19: error: reference to 'integral_constant' is ambiguous
46 | const _Tp integral_constant<_Tp, __v>::value;
|
This failure appears to trace to
std::string i_ddLevelStr; //Same
which is used as
ddLevel = strtol(i_ddLevelStr.c_str(), NULL, 16);
or similar for a total of two assignments and four references. Again, the word "lazy" comes to mind.
There are deeper uses of C++, of course, such as
numRingIds = P9_RID::NUM_RING_IDS;
The namespace for P9_RID can be found in
src/import/chips/p9/utils/imageProcs/p9_ringId.C
I do not see anything that could not have been accomplished with C99 or later. I suspect these files were designed to be compatible with AIX and IBM's compiler, not to be portable. I have never seen any C++ code that could not be accomplished with C and a simpler compiler.
There are 391 files in the SBE that have C++ :: operators, totaling 8,178 lines. I do not see this as being readily portable to C, and as a result of conflicts with other operating system and compiler implementations of C++, not really portable at all.
The native Linux build environment for the SBE can not compile successfully. I suspect that within Raptor the build environment has been steadily patched and an attempt to build from scratch has not been undertaken in some time. I suspect that even simple changes upstream could have a devastating effect on Raptor, should it occur in certain places or if certain people leave (and their knowledge with them). This is an unfortunately common experience, and in large software efforts can lead to terrible inertia.
The sheer complexity of the boot process and dependency on external parties leaves me with no choice but to stop development. This is not an easy decision to make. I am partial to PowerPC, and enjoy working so close to bare metal. However, I can not see a path forward while maintaining the simplicity I am trying to implement.
I am deeply concerned about the performance limits with the RISC-V chips I have seen, but their simplicity and low cost offers the hint of a path forward. Personally, I would really be excited to see Raptor hardware running multi-core RISC-V, if the "open" philosophy can be maintained. I am deeply concerned that many of the major players in the RISC-V and Open Compute sphere are the very same players that have undermined openness in tech for decades. I rarely find a RISC-V SBC that isn't actually tied to blobs required to run the proprietary peripherals, and even entire toolchains are vendor-dependent. Don't even think about asking if the firmware can be replaced.
As I previously noted in another post, the SG2000 chip was actually a surprisingly good performer for the 1w of power it consumed, although something was clearly impeding it, possibly the i- and d-caches being very, very small. However, Pine64 and Milk-V were able to deliver boards for as little as $13.
Say it was possible to scale up to a 64 RISC-V core like the SG2000 (which I think is embargoed in the U.S. now), and it consumed 64 watts at peak power. From my (homemade real-world integer-based) benchmarks, it would take 1.4 times to complete a parallel task as would a 16 core Raptor system (running SMT4), at half the power consumption (each one takes 76 times the elapsed time on the 64 thread Raptor system). To me this hints of a cost-effective opportunity to scale multi-CPU, with some hardware enhancements. Just a thought.
Again, my many thanks to Raptor CS and their efforts.
tim
I wanted to again express my appreciation to Raptor CS for access to the hardware. I used that access to do a lot of bare metal testing as well as access to the lowest levels of the firmware, in an effort to build a truly portable build environment. This has been a fantastic learning experience. I sent the following summary to Raptor a few months ago, but then took some time off from computers and failed to post the summary to the forums (unless my memory is worse than I remember).
I have done a lot of digging into the internal boot process, specifically the SBE, and while in theory the boot process can be replaced, in practice this appears to be extremely labor intensive. I have been working with Public Domain POSIX Make
https://frippery.org/make/
to rewrite the Makefiles to be strict POSIX 2024. The GNU syntax in the Makefiles ties the build process very closely to GNU toolchain. They are not compatible with FreeBSD's make. The solution is strict POSIX 2024, but there are 28 Makefiles and 30 .mk files that need to be rewritten.
Note that strict POSIX 2024 does not support out-of-tree building, where the source code is in one directory and the object files are in another. The main cause (effect?) of this is that implicit target rules can not use variables, so $(OBJDIR).o: is not a valid target. The IBM writers of the SBE Makefiles leaned very heavily on GNU's extensions, to the point where I would describe the effort as "lazy." I suspect at least a few of the Makefiles were generated by scripts.
I have rewritten several Makefiles, enough to dig deeper into the SBE build process. The C++ implementation/includes on FreeBSD are not compatible with Linux's C++ implementation. The naming for #ifndef is different, leading to duplication of std functions. In a particular case, xip, the two source files are .C, but one of them does not contain any C++ while the other only contains five lines total, out of 3,237 in the file. That file fails to compile because of the duplicate:
In file included from p9_xip_tool.C:41:
In file included from /usr/include/c++/v1/string:591:
In file included from /usr/include/c++/v1/__algorithm/remove.h:12:
In file included from /usr/include/c++/v1/__algorithm/find.h:16:
In file included from /usr/include/c++/v1/__bit/countr.h:15:
In file included from /usr/include/c++/v1/__bit/rotate.h:15:
In file included from /usr/include/c++/v1/limits:581:
/home/tkelly/raptor/talos-sbe/src/import/chips/p9/procedures/ppe/pk/../include/std/type_traits:46:19: error: reference to 'integral_constant' is ambiguous
46 | const _Tp integral_constant<_Tp, __v>::value;
|
This failure appears to trace to
std::string i_ddLevelStr; //Same
which is used as
ddLevel = strtol(i_ddLevelStr.c_str(), NULL, 16);
or similar for a total of two assignments and four references. Again, the word "lazy" comes to mind.
There are deeper uses of C++, of course, such as
numRingIds = P9_RID::NUM_RING_IDS;
The namespace for P9_RID can be found in
src/import/chips/p9/utils/imageProcs/p9_ringId.C
I do not see anything that could not have been accomplished with C99 or later. I suspect these files were designed to be compatible with AIX and IBM's compiler, not to be portable. I have never seen any C++ code that could not be accomplished with C and a simpler compiler.
There are 391 files in the SBE that have C++ :: operators, totaling 8,178 lines. I do not see this as being readily portable to C, and as a result of conflicts with other operating system and compiler implementations of C++, not really portable at all.
The native Linux build environment for the SBE can not compile successfully. I suspect that within Raptor the build environment has been steadily patched and an attempt to build from scratch has not been undertaken in some time. I suspect that even simple changes upstream could have a devastating effect on Raptor, should it occur in certain places or if certain people leave (and their knowledge with them). This is an unfortunately common experience, and in large software efforts can lead to terrible inertia.
The sheer complexity of the boot process and dependency on external parties leaves me with no choice but to stop development. This is not an easy decision to make. I am partial to PowerPC, and enjoy working so close to bare metal. However, I can not see a path forward while maintaining the simplicity I am trying to implement.
I am deeply concerned about the performance limits with the RISC-V chips I have seen, but their simplicity and low cost offers the hint of a path forward. Personally, I would really be excited to see Raptor hardware running multi-core RISC-V, if the "open" philosophy can be maintained. I am deeply concerned that many of the major players in the RISC-V and Open Compute sphere are the very same players that have undermined openness in tech for decades. I rarely find a RISC-V SBC that isn't actually tied to blobs required to run the proprietary peripherals, and even entire toolchains are vendor-dependent. Don't even think about asking if the firmware can be replaced.
As I previously noted in another post, the SG2000 chip was actually a surprisingly good performer for the 1w of power it consumed, although something was clearly impeding it, possibly the i- and d-caches being very, very small. However, Pine64 and Milk-V were able to deliver boards for as little as $13.
Say it was possible to scale up to a 64 RISC-V core like the SG2000 (which I think is embargoed in the U.S. now), and it consumed 64 watts at peak power. From my (homemade real-world integer-based) benchmarks, it would take 1.4 times to complete a parallel task as would a 16 core Raptor system (running SMT4), at half the power consumption (each one takes 76 times the elapsed time on the 64 thread Raptor system). To me this hints of a cost-effective opportunity to scale multi-CPU, with some hardware enhancements. Just a thought.
Again, my many thanks to Raptor CS and their efforts.
tim
