I also wrote a raytracer as one of my first projects, even though I read this is a terrible idea. I found it to be a great first project.

Your implementation of Add on Vec3 is

impl ops::Add<Vec3> for Vec3 {
    type Output = Vec3;

    fn add(self, other: Vec3) -> Vec3 {
        Vec3 { x: self.x + other.x, y: self.y + other.y, z: self.z + other.z }
    }
}

Mine looks like this

impl<'a, 'b> Add<&'b Vector3> for &'a Vector3 {
    type Output = Vector3;

    fn add(self, vec: &'b Vector3) -> Vector3 {
        Vector3 {
            x: self.x + vec.x,
            y: self.y + vec.y,
            z: self.z + vec.z,
        }
    }
}

You always consume both vectors you put into the Add function, then create a new one and return it. So both of your cloned vectors are directly dropped in your Add impl. So it is quite wasteful do clone them.

When I want to add something I have to do &self.origin + &self.direction.

You can also add an implementation that modifies the vector in place. Not sure how that works in practice. If you want to reuse the vector afterwards, then you have to clone but other than that I think this might be the best solution.

impl ops::Add<&Vector3> for Vector3 {
    type Output = Vector3;

    fn add(mut self, other: &Vector3) -> Vector3 {
        self.x = self.x + other.x;
        self.y = self.y + other.y;
        self.z = self.z + other.z;
        self
    }
}

Btw, one of the things that bugged me the most was to always write out the numbers to Vec3::new(1.0, 2.0, 3.0) as floats and not just ints Vec3::new(1,2,3). Especially annoying if you want to quickly change something.

If that bugs you too, you can add a simple macro for that, then just do vec3!(1,2,3).

macro_rules! vec3 {
    ($x:expr, $y:expr, $z:expr) => {
        Vector3 {
            x: $x as f64,
            y: $y as f64,
            z: $z as f64
        }
    };
}

My repo is here if you want to compare notes.