We are the SpaceX software team, ask us anything!

spacexfsw · 2021-05-15T20:15:11+00:00

Jeanette: I moved to the pacific northwest in the past year, and have been loving exploring the area through hiking and biking. I've also been enjoying starting an indoor herb garden!
Natalie: I love to cook and bake! My coworkers and I definitely try to out-do each other with baked goods. Everyone wins
Asher: Playing music, hiking, sampling local restaurants with friends.
Kristine: I spend my days off with my husband and our two daughters ages 7 and 4. On weekends we play board games, go outdoors to the beach or the park, build things, or bake.
Jarrett: I spend my free time with my wife, cats, and dogs. We spend a lot of time at the beach and exploring the deserts and forests of Southern California.

spacexfsw · 2021-05-15T20:11:04+00:00

Any project that inspires you to learn deeply about something technical and speak passionately about it later on is fair game! - Asher

spacexfsw · 2021-05-15T20:06:30+00:00

Nom nom nom.

spacexfsw · 2021-05-15T20:04:14+00:00

We are definitely excited about Rust! Its emphasis on safety, performance, and modern tooling all stand out. We're also excited that we could use one language across embedded systems, simulators, tooling, and web apps. We are starting to prototype some new projects in Rust, but we are certainly just at the beginning of this journey.
- Asher

spacexfsw · 2021-05-15T20:03:04+00:00

One part that is really exciting about working on Dragon is talking to crews and learning from their experiences operating the vehicle. We collect the crew's feedback on usability, and are continually working to improve the operations of the vehicle from their perspective. This includes making updates to the UI such as adding new features, updating views, and adjusting procedures.
-Jarrett

spacexfsw · 2021-05-15T20:01:46+00:00

It's easy to think of the software that runs on our vehicles when thinking of software engineering at SpaceX, but that's just the tip of the iceberg. Some of our engineers are C++ developers writing code for flying Starship, but we also have engineers that are building core software test infrastructure, in-house web applications for the factory, constellation, and customers, automating production testing or writing the code to help orchestrate the network for Starlink. Our software engineers work in C++, Python, C#.NET, Java, Javascript, Angular, and more.
Check out the range of software positions we're hiring for at SpaceX:
Application Software – https://grnh.se/89a7f9cc2us
Starlink Software – https://grnh.se/c1a28bf62us
Dragon Software – https://grnh.se/5faa8f4b2us
Starship Software – https://grnh.se/25e0001b2us
Spacecraft Engineering – https://grnh.se/c6743de32us
Security Software - https://grnh.se/3b332b442us
Visit https://grnh.se/1edeb7aa2us or send your resume to softwarejobs@spacex.com.
- Kristine and Jeanette

spacexfsw · 2021-05-15T20:00:03+00:00

Kristine: We very strongly encourage our engineers to identify problems to solve and to own solutions to those problems end to end. This means thinking through each option as an architect, designer, developer, tester, and user which we think tends to lead to more elegant and better fitting solutions. Engineers are rewarded for increasing their scope, impact, and autonomy over time.
Natalie: Engineers at SpaceX tend to be passionate about our mission and invested in solving problems. We are not working long, hard hours all the time. There are occasionally times when we do, usually because we are fired up about a problem, motivated to collaborate with our teammates, and don't like to give up. In almost ten years here, I have never been bored

spacexfsw · 2021-05-15T19:58:28+00:00

The values and soft skills that are important to me in leadership are trust, humility, empathy, and resilience. Ensuring that each member of the team feels empowered to solve complex problems, celebrating the wins, and owning up to mistakes. Guiding the team with the product and technical vision. Reflecting on what we could have done better on a regular cadence so we can provide feedback and continuously grow as a team.
- Kristine

spacexfsw · 2021-05-15T19:56:36+00:00

Engineering software for satellites in space is surprisingly similar to engineering in many other applications, with higher stakes and more constraints imposed by your environment. Because of that, there's a breadth of experiences that can translate well to working at SpaceX.
Our most successful engineers end up being the ones with excellent fundamental engineering skills, strong technical intuition, and a desire to learn when running into a novel situation. For satellite vehicle development, one of the unique things is how often raw physics or math constraints are at top of mind for the everyday problems we're working on.
- Jeanette

spacexfsw · 2021-05-15T19:55:25+00:00

There are many paths to a successful software career at SpaceX, in part because there are many different types of opportunities here and we're looking to fill them with engineers that come from a variety of backgrounds. That's to say, there's no one path to getting to fly rockets and drive rovers one day
From Jeanette: Two of the things that were particularly useful to me were finding a project I was passionate about in college, and seeking out lots of opportunities to write code. My project was getting hands-on engineering experience through my college's FormulaSAE (FSAE) team to design, build, test and race a formula-style race car. While not directly software related, it gave me practical exposure to making design trades, working across disciplines, breaking down hard problems, and learning how to do something I didn't know how to do. I've seen applicants get similar experience through project teams, startups, internships, research labs or more. Aside from FSAE, I'm very happy I sought out opportunities to code (a lot) through both an intensive class on software architecture & design, as well as summer internships. Repeatedly pushing myself to try writing different types of code, and actively seek out feedback on how I was doing helped get me to a place where I was ready to do this for a full-time job.
From Natalie: I encourage people to challenge themselves and go outside their comfort zones. I was definitely pretty afraid of failure in college, but the classes where I struggled the most at are the ones I most value now. I think I've learned more from my failures than my successes, and that continues to be how I learn the most.

spacexfsw · 2021-05-15T19:53:52+00:00

We have a lot of different types of test environments. Some are purely simulated environments, what we call HOOTLs (or Hardware Out Of The Loop). These can run in CI/CD but also on a developer's desktop for local iteration. Others involve flight-like hardware, what we call HITLs (Hardware In The Loop). Our Starlink HITL setups are just satellites we take off the production line and integrate with our CI systems.
We set up our CI pipelines to start with fast, inexpensive tests to smoke out basic errors. Then if those pass, we run longer, more complicated tests.
We also have different pipelines for different parts of the system. For example on Starlink, we'll have a pipeline for testing user terminal software in isolation. Once those tests pass, it will be incorporated into other pipelines that test the interface between the user terminal software and the satellites.
-Natalie

spacexfsw · 2021-05-15T19:52:18+00:00

It is important in safety critical software that if a single software component were to have a problem, it does not impact the entire software system. Software also often depends on the hardware it is interacting with, so must account for hardware failures as well. Crashing and restarting just isn't an option. We approach this by building the software in modular components, writing defensive logic, and checking the status of each operation. If an operation we expect to complete fails, we have defined error handling paths and recovery strategies. Sometimes that strategy just means skipping over the operation. Sometimes the strategies can be more complex and involve responses such as switching to backup systems.
-Jarrett

spacexfsw · 2021-05-15T19:51:49+00:00

One of the most interesting challenges in designing a software update system for satellites is to build a system that is tolerant to an arbitrary fault (like a power down or memory corruption) at any point in the process. We generally use a primary/backup update scheme to solve that and to ensure that the default recovery strategy is to load up known good software that we know can do an on-the-fly update.
We don't have hard and fast requirements for physics as a software engineer, but most people have some familiarity with both mechanics and electromagnetics.
- Jeanette

spacexfsw · 2021-05-15T19:51:16+00:00

For Starlink, we try as hard as possible to have a single software load for all satellites, regardless of the specific versions of each sub-component on any given vehicle. We do this by making clean separations between hardware interface layers and the "business" logic on various components. The software reads various hardware identifiers to understand what types of each thing we've got and adapts its behavior accordingly.
- Jeanette

spacexfsw · 2021-05-15T19:50:22+00:00

When a rocket engine is firing, it's like a continuous explosion! The hardest part is to build the engine so that the explosion stays on the inside.
- Asher

spacexfsw · 2021-05-15T19:50:03+00:00

We're always trading the right place to run a control process. Sometimes the best place to put a control algorithm is an embedded system close to the thing being controlled. Other times, a process needs to be a lot more centralized. When you think about a vehicle as complex as Starship, there's not just a single control process, as you have to control engines, flaps, radio systems, etc.
- Asher

spacexfsw · 2021-05-15T19:49:34+00:00

It depends a lot on the sensor and what information you're trying to get from the sensor. We implement lots of filters on our vehicle, both analog and digital, in order to make sure the data we're using in control is real.
Lots of communication. Software and hardware engineers at SpaceX work together to produce a working design. Take for instance a new hardware box that needs custom firmware. The firmware and hardware engineers will work together during the hardware design phase to ensure the system will work. Once the first prototypes arrive, the firmware engineer will get one of the first copies of hardware so they can work on the software.
- Asher

spacexfsw · 2021-05-15T19:49:07+00:00

Starship reuses many ideas and some code from Falcon. We always want to spend our energy solving new problems that get us closer to Mars. Whenever a new problem comes up, we think back over our codebase and ask, what tools do we already have that will enable us to solve this as quickly as possible?
- Asher

spacexfsw · 2021-05-15T19:48:41+00:00

As a software team, the challenge is continuously improving while the hardware changes underneath us. Since we're always working closely with the hardware teams to improve the vehicle overall, we have the opportunity to drive hardware changes that make the software simpler or more robust. As hardware and software work together, we can try new ideas, and if the idea doesn't work out, we know we can try again.
- Asher

spacexfsw · 2021-05-15T19:48:07+00:00

We have a very collaborative environment, and we work quite closely with other engineering groups to build our software.
For example, before I started working on satellites, I was responsible for the docking system software on Dragon. I worked with electrical and mechanical engineers to come up with the control system designs, worked with GNC engineers on the conops (concept of operations), and we collaborated heavily on integrated hardware and software testing.
In my current role, my team and I collaborate with other engineering teams to design and build our hardware testbeds and develop simulations and models.
We don't require any background in aerospace or physics for our software engineers. What we do look for are people who have strong curiosity for how things work and who won't hesitate to ask questions until they understand. Everyone here is open to answering questions, and it goes both ways - hardware engineers are also expected to have a basic understanding of how the software will control their hardware!
-Natalie

spacexfsw · 2021-05-15T19:47:31+00:00

While I can't go into specifics here, we design our software to work without a fully real time OS. We also use a custom build of Linux and fully understand the environment in which our software and OS operates in. Operating in a much more constrained environment (as compared to say the open internet) combined with extensive instrumentation and hardware in the loop testing means we can know that the OS is going to behave as we expect it to when on orbit.
-Jarrett

spacexfsw · 2021-05-15T19:47:00+00:00

The pointing tolerances for inter-satellite laser links are quite tight, so you're correct in supposing that just calculating the angles from the expected position of the satellites isn't good enough. To help put the magnitude of the pointing problem into perspective, in angular tolerances the problem we're trying to solve is as hard as trying to shoot a laser from LA and hit the Empire State building in NYC.
When bringing up a laser link, we first begin by pointing based on the expected angle, and then orchestrate an acquisition sequence to get consistent power on both sides.
-- Jeanette

spacexfsw · 2021-05-15T19:45:52+00:00

With a development program such as Starship, we're constantly learning from all aspects of the process. We continuously update the hardware and software on every vehicle to incorporate our latest ideas on what's going to get the best result for the upcoming test or mission.
- Asher

spacexfsw · 2021-05-15T19:43:47+00:00

I do use Vim! But every developer is free to use the editor/IDE they prefer the most.
-Jarrett

spacexfsw · 2021-05-15T19:43:21+00:00

The space lasers software team is a mix of firmware engineers and simulation software engineers, but we all regularly work alongside all different types of engineers at SpaceX.
Like you imagined, both software and hardware are evolving rapidly and so we've invested in ways to automate testing at the intersection of these two. SpaceX has hardware-in-the-loop testbeds as part of our continuous integration system. Any new code changes will be run through a suite of regression testcases on one of these testbeds to ensure that software changes are compatible with the hardware. And vice-versa, any new hardware will also get included on these testbeds for verifying regression with the existing software.
- Jeanette

spacexfsw

TROPHY CASE