At the dawn of the Space Race, when computers were something that took up whole rooms, satellites and probes had to rely on analog electronics to read from their various sensors and transmit the resulting data to the ground. But it wasn’t long before humanity’s space ambitions outgrew these early systems, which lead to vast advancements in space-bound digital computers in support of NASA’s Gemini and Apollo programs. Today, building a spacecraft without an onboard computer (or even multiple redundant computers) is unheard of. Even the smallest of CubeSats is likely running Linux on a multi-core system.

As such, software development has now become part an integral part of spacecraft design — from low-level code that’s responsible for firing off emergency systems to the 3D graphical touchscreen interfaces used by the crew to navigate the craft. But as you might expect, the stakes here are higher than any normal programming assignment. If your code locks up here on Earth, it’s an annoyance. If it locks up on a lunar lander seconds before it touches down on the surface, it could be the end of the mission.

To get a bit more insight into this fascinating corner of software development, we invited Jacob Killelea to host last week’s
Software for Satellites Hack Chat. Jacob is an engineer with a background in both aero and thermodynamics, control systems, and life support. He’s written code for spacecraft destined for the Moon, and perhaps most importantly, is an avid reader of Hackaday.

RELIABILITY ABOVE ALL ELSE

The conversation started about as you’d expect, with several people wanting to know what kind of languages, frameworks, and even operating systems are used in today’s spacecraft. Jacob says while there’s an incredible amount of variability out there depending on the hardware and what the software needs to do, much of it is familiar to folks like us. He says the language of choice tends to be C, and that while Linux is used, it tends to be for higher level tasks that don’t need to happen in real-time. If not running on the bare metal, critical code is likely to be running on something like VxWorks. Although even here he cautions that the aerospace community prefers to stick with what works, so you may find the spacecraft you’ve been tasked to write code for is using an OS from the early 2000s.

Reliability is ultimately the name of the game when writing code for space applications, which brought the conversation towards fault tolerance and so-called “safe mode” operation. Given that faults can be triggered by external events outside of your control (such as cosmic rays), even the most carefully crafted and tested code may one day crash. In that event, there needs to be a secondary system that can take over and put the craft into a known-good state. Interestingly, these “safe-mode controllers” are often a dedicated module and not just a different operating mode of the main computer.

This provides true redundancy in the event of a complete computer failure, but isn’t without its own risk — Jacob recalled a mission he’d studied where a controller designed for a previous vehicle had been reused on one with a different physical layout. Everything was fine until the satellite eventually went into safe-mode, at which point the controller’s attempts to stabilize the craft actually caused it to tumble out of control. In the end the “safe-mode” ended up being anything but, and the vehicle was lost.

TEST LIKE YOU FLY

Others in the chat wanted to know about what kind of simulations or testing can be done with spacecraft software here on the ground. Jacob says one of the most powerful tools is what’s called a FlatSat, which could be considered something akin to the breadboard version of the spacecraft’s final hardware. All of the craft’s electronic components are laid out on a workbench, with ample test-points that allow tapping into the power and communication busses. This gives engineers the necessary access to test different modules and simulate various failures in ways that would be difficult or perhaps even impossible if using a replica of the flight hardware.

But still, this only gets you so far. Jacob points out that it’s all but impossible to accurately simulate the space environment. There’s no way to create microgravity in the lab, so how will you know how it impacts your inertial measurement units (IMUs) in orbit? How can you be sure your optical star tracker won’t be confused as the craft rotates when your test rig is bolted to the table? The answer is, simply, that you can’t. All you can do is test for as many edge cases as you can think of, and make sure the system can fail as gracefully as possible.

That said, you can get an idea of how your electronics will respond to cosmic radiation without hitching a ride to space. Jacob says he was involved with a project where they tested their system against radiation induced faults by putting it in the cyclotron at Texas A&M University and hitting it with proton beams. This produced numerous fault conditions, some of which they were able to devise recovery procedures for. Of course there’s no way of predicting what will actually happen in space, and satellites in lower orbits are partially protected from radiation by the Earth’s magnetic field, but it’s still good to at least have an idea of what you’re up against.

ON THE SHOULDERS OF GIANTS

In decades past, each spacecraft or mission was an entirely new venture — a real “to boldly go where no one has gone before” kind of thing. But today’s aerospace engineers have the benefit of shared spacecraft platforms and robust software frameworks that allow them to incorporate the hard-learned lessons of previous missions into their new craft.

For example, Jacob strongly recommended anyone interested in spacecraft software take a look at NASA’s Core Flight System (cFS). This Apache-licensed flight software framework can run on Linux and various real-time operating systems (RTOSs), and provides modules for various spaceflight-related tasks such as telemetry, process management, error reporting, command scheduling, etc. Even if your homebrew rover isn’t likely to get much farther than the back garden, it could probably benefit from some of the research and software development that NASA has already done.

Jacob also mentioned that more modern satellites, especially the larger and more capable variants that operate in deep space, have enough computing power that they can offer up virtualized environments for “software payloads” that are uploaded from the ground. So for example if you had some kind of research you wanted to conduct using the sensors on a given spacecraft, you could potentially upload your own code that would run in a protected environment where failures won’t endanger the craft’s vital systems.

This is especially helpful on spacecraft designed for exploration or scientific observations. It turns out the James Webb Space Telescope (JWST) uses a system like this so scientists that “visit” the observatory virtually can operate its instruments safely through programs written in, of all things, JavaScript.

Jacob was good enough to end the chat with a rundown of various systems and technologies at play in modern spacecraft, such as standardized data busses or telemetry protocols. Basically he took the time to answer the questions that most of us aren’t well versed enough to even ask, and the results were absolutely fascinating. Like they say, you don’t know what you don’t know.

We’d like to thank Jacob for giving us a taste of what it’s like to develop software for modern spacecraft. While we might not get the chance to run any of our own code in orbit, we can all certainly benefit from some of the lessons learned while operating in such a hostile environment.

by: Tom Nardi for hackaday.com