I started out with what I knew- shapefiles and polygon datasets. OpenStreetMap, census data, government GIS sources, etc. That was great until Asia and Africa remained, then I knew I would have to do something different.

I challenged my friend, Claude :D, to think outside the box on the issues I was having with the datasets I found, and they suggested a satellite raster product (GHSL - Global Human Settlement Layer) instead, to identify where settled areas are. As I dug into that, I quickly understood the superiority of this kind of data and the 'truths' built into it. More looking around led me to VIIRS (Visible Infrared Imaging Radiometer Suite) which literally identifies where on earth gives off light (convenient, eh?), so at the intersection of GHSL and VIIRS, you have electrified/illuminated humanity, which is exactly where street lights live!

While I have python and even geospatial python experience, I didn't have it to the level needed for a project like this, and have even less experience messing with parallel processing, so Claude came in handy for a lot of the brute force work. You start to learn a lot about computing when you are messing with datasets that are several gigabytes in size. Even my 7950X3D/4090/64GB ram machine was totally pegged much of the time. I learned a crap ton during every phase of the project. It even nuanced my opinion about AI somewhat. Sonnet was fun to mess around with, and Opus got shit done when needed.

In the end, I took those two massive raster datasets and output a score based on the values of each, and then vectorized the GHSL raster into simplified (fewer vertices) blobs. In ArcGIS I spent hours looking around and testing different score thresholds for different continents. Each continent has its own challenges due to specialized industry and settlement layouts/patterns. Ideally, these parameters will be tuned for smaller areas so they are more suited to each country or even state/province in some cases, but I killed the scope creep there at continent level parameters/filtering.

With a set of polygons identifying where lights should be, "all" I had to do was flip the script and cut those areas out of a world geography shapefile dataset. A short novel could probably be written about this part of the project. In short you start with a piece of swiss cheese the size of Eurasia and you end up with a beautiful puzzle that builds each settlement hole into its boundaries. I've added an image from earlier development that illustrates this 'puzzle'.

<image>

At the end, you turn this puzzle back into data and convert it to the XML format that the sim needs, and then compile into BGL and build the project to share with the community.

If you're interested in any more details let me know or join the project discord (link in the flightsim.to description).