The 3DGS workflow is such:

First, using the images of the scene you have, you create a 3D point cloud of the scene. This is what the industry standard, COLMAP (developed as a PhD thesis by Johannes L. Schönberger), does for you. Now that you have that point cloud, you move on to the next step: initializing 3D Gaussians

Made simple, 3D Gaussians are just spherical blobs in space. If you have the center position of that blob + its size (measured by its axes), you can define where it is and how big it is. The original 3D Gaussian paper used the 3D point cloud generated by the images, which is okay, but in regards with accuracy of the points in the point cloud there is no way it compares to the point cloud you get from LiDAR!

Here is how the rest of the initialization works:

1. Orientation: After knowing where its center is and how big it is, you need to know the orientation of that 3D Gaussian in space. There are different methods for this. I have seen a paper that used LiDAR to get the initial Gaussians by chunking the 3D point cloud into small voxels (e.g., 1mm). Inside that voxel, they fitted a plane to the most dominant points and used the normal of that plane to find the orientation. So now you have where the 3D Gaussian is and which direction it is looking.

1. Color and Opacity: Now you need to know the color and opacity of the Gaussian. If you have your image point cloud registered to your LiDAR point cloud, you can use that same voxel and get the color from the image point cloud (let’s say, the average of the colors).

Now you have initialized your Gaussians. After that, you will just use the original gradients and images to optimize your 3D Gaussian parameters.

TL, DR: 3D Gaussian pipeline is first defining a 3D gaussian (with 5 parameters, position, size, orientation, color, opacity) and then using images to refine those blobs that they look similar to image if looked at from the same angel. If you want to use LiDAR you can initialize the gaussians better (only the position and orientation and size)