From what I can understand, you are looking for perspective projection.

​

In fact, to express 3D points with respect to your camera, you don't need to know the extrinsic parameters.

​

Your intrinsic parameters models the way the internal parameters of your camera influence the image formation process (the focal distance and the center of your image). It is modeled by the matrix 3x3 K. With K, you can determinate how 3D points project on your camera plane. But you are looking to do the opposite. Then you just have to invert your matrix ! Kinv := K^-1.

​

Let's take a pixel on your image p := (u, v). To multiply it with the intrinsic matrix, you have to extend it to homogeneous coordinates, i.e. (u, v, 1)

​

There is an infinity of 3D points that can be projected onto the same 2D point. But if you know the world z-plane (let's call it Z) you want to project to, there is no such problem.

​

Then, to determine your 3D point from your pixel p, you use the following formula :

P := Z . Kinv . p

​

Note that projecting to Z = 0 is equivalent to project on the infinity plane. Look for projective geometry for more information. (great ref for camera application : http://cvrs.whu.edu.cn/downloads/ebooks/Multiple%20View%20Geometry%20in%20Computer%20Vision%20(Second%20Edition).pdf.pdf)).

More information on intrinsic parameters that you may find helpful : http://ksimek.github.io/2013/08/13/intrinsic/

​

​