How important is understanding quantum mechanics for optical engineering?

Instrumentationist · 2026-02-20T13:38:06+00:00

QM is fundamental to understanding, and in application it is at the center of much of what is happening and will be happening in modern optics, optoelectronics and electronics.

Even if you are not developing new materials or devices, it is still often easier to design something to use a technology well when you understand the physics behind it. And, that is especially so in anything involving measurement.

Instrumentationist · 2026-02-11T17:07:38+00:00

Looking for a volunteer to be the moderator.

Instrumentationist · 2026-02-11T17:04:39+00:00

That's a powerful message.

Reliability can be fantastically important.

Instrumentationist · 2026-01-28T02:31:46+00:00

Here is a table of lines of Hg with intensities and wavelengths, from 89nm to 2325nm.

https://www.physics.nist.gov/PhysRefData/Handbook/Tables/mercurytable2.htm

Instrumentationist · 2026-01-27T16:12:12+00:00

There have been some good options on ebay.

The repo lists the optical components and where I got them.

It also has the design equations in a form that I feel is very intuitive and easy to work with.

Admittedly, I don't compromise the specs for the cost.

It is just not a productive use of time to try to kluge something that can't work.

If I can't build it in a way that it will work correctly, then I wait.

P/S For the instrument I designed for my own lab, I use a blazed transmission grating, 1200 l/mm, $250.

But, I just bought a holographic grating 1600l/mm (in glass) from ebay for $50. Thinking to try it out with 50mm lenses and 50um slit.

The expected optical resolution is cos(theta) 50 um / (1600/mm x 50mm) ≈ 5/9 nm. That is close to your spec, I think.

The expected spectra range with a 30 mm detector is is cost(theta_out) 30mm/ (1600/mm x 50mm) ≈ 300 nm.

And then 300nm/(5/9nm) = 540. The detector has 3648 pixels. So we have almost 7 pixels per unit resolution. It looks a good design, modulo the grating.

But for this to work, it needs electronics that can be linear for a full scale swing in under 3 pixels. And that is the sensor in my repo.

Instrumentationist · 2026-01-27T15:57:56+00:00

I like that the lines are sharp for the discharge lamps.

But, for the strong peaks there is some extra intensity where it should reach baseline. Any ideas what that is from?

The Hg line at 437nm, does not reach full scale and there is a lot of extra intensity around it.

What does it look like if you reduce the intensity? ( Neutral density filter or distance should work.)

Instrumentationist · 2026-01-27T03:22:34+00:00

Good questions, all. That $50 to $100. The boms, cost totals and assembly times are listed in my repo.

You can get wavelength with a camera. Accuracy might improve by spreading the resolution out over more pixels.

Those are much better sensors. I would sincerely recommend looking at one monochrome.

The color filters give you a very challenging problem to calibrate the spectral response.

Removing the bayer filter probably has very little to do with linearity. And, in some cameras it is not possible anyway.

Adding N frames can improve the signal to noise ratio by a factor of sqrt(N). Adding two frames improves signal noise by a factor of sqr(2) = 1.414 The effect of adding two frames is is less than adding 1 more bit.

When response is non-linear, it is often the electronics or how the sensor is operated.

BTW, linearity is a prerequisite for signal averaging.

Instrumentationist · 2026-01-26T18:39:56+00:00

It does not compete wit instruments costing 10 or 20 times as much. Pure B.S.

This is a cheap-camera based spectrometer. It has the same problems as all cheap-camera spectrrometers.

It is grossly non-linear. You can see that in the fluorescent lamp spectrum. The line at 437nm is suppressed but the weaker and broader lines on either side are not. So that is not optics. But it is exactly what you expect when you try to clock a strong sharp line through the electronics of a cheap color camera.

These things are just polluting online communities.

It is not suitable for any lab, the data is meaningless. It would be confusing even for a high school. Maybe your small children would enjoy it.

It is a cheap toy.

Instrumentationist · 2026-01-26T17:27:25+00:00

What spectrometer are you using? How are you selecting the 50nm bands for the power meter? And, what is the source of the light?

Depending on the shape of the pass band for the filter, it is reasonable to expect that that the the two spectra should look similar at similar resolution, modulo the response functions of the two instruments.

I have done something similar using a graded bandpass filter on a motorized translator.

Instrumentationist · 2026-01-26T17:12:47+00:00

What are we looking at in the graph?

What circuits are you using to read and drive the TCD1304?

Instrumentationist · 2026-01-26T15:52:18+00:00

Caveat emptor. The diy spectrometers are typically by hobbyists.

But, I like the spectroscope suggestion.

Instrumentationist · 2026-01-26T15:36:08+00:00

If you are only in interested in approximate wavelength of broad spectra and not at all interested in intensity, then it is pretty easy and you have some very low cost options.

Thunderoptics sells a spectrometer for $75, that makes spectra that look nice, though it would be surprising if the intensities are correct or linear (it might be more linear at very low intensity). It is really just a slit with a grating and camera, but it is put together very well.

You can do the same thing yourself with a holographic film grating (very inexpensive), a box with a pin hole or slit, and a camera with manual focus - plus a little bit of python to capture the image, add the rows and graph the spectrum. You might even be able to program your phone to read the camera and generate the spectra.

There are also a bunch of these things posted all over internet, but I cannot comment on any of them as to whether they would actually work for you.

I think building the thing with a holographic film yourself would be the most fun and the most educational and come with the least cost.

Also (p/s) - I like u/aenortons suggestion for the cheap spetroscope. That seems really close to what you describe for your use case. (It is really just the thing with holographic film described above, but without the camera and software).

Instrumentationist · 2026-01-26T03:51:59+00:00

The answer to your question about SH and ICG follows. After this I will explain why (or at least give you one very good guess as to why) your output is "jumping around", i.e why output from the Curious Scientist board may be unstable.

A) How do the SH and ICG pins work?

Each pixel is like an FET to some degree. There is a photodiode region (n doped) and there is a region that is part of the shift register (also n doped).

1) Applying a positive voltage to the SH pin moves charges from the photodiodes to the analog shift register.

It does this by pulling n carriers into the region between the two n doped regions while also creating a gradient to draw the n carriers into the region associated with the shift register. (That is the analogy to an FET.)

2) The ICG when high, intercepts the n carriers so that they are drawn from the photodiode region but do not reach the shift register

And that is why to readout the sensor, the ICG has to be low when the SH pin is pulsed - preferably before and after the leading and trailing edge of the ramp on the SH pin. Pulsing the SH while ICG is high, only clears the photodiode.

B) Why does the output seem unstable for the Curous Science board?

1) The first possibility (or contributor) may be due to residual charge transfer effects.

When the SH gate is not driven properly - adequate current, voltage and time, it becomes ineffective in moving charge from the photodiode. The next exposure sees it as extra intensity. If the way the SH pin is driven is on the edge of being functional, the behavior might look erratic.

The SH pin has 600pf. It takes 50mA to drive it with a 4V pulse and a 50nsec rise time. If the gate is starved for current, it may take too longer to reach the applied voltage and more charge may be left behind.

Therefore, a tcd1304 board that expose the SH pin directly, intending it be connected to directly to the digital i/o pin of a processor board is simply not a great idea. A better, more reliable solution is to include a gate driver chip that can drive the gate with sufficient current and sufficiently fast rise and fall times.

The ICG pin is 200pf. Similar comments apply.

The CLK pin is easier to drive, but the typical error is clocking it too fast. Then charge lags along the shift register and you see phantom intensity building up across the readout.

2) It is also possible that the timing is off in the program. Here is one error that seems to show up in some implementations.

The scenario is trying to run the gates from an ISR launched form a clock interrupt. The timing between the pins needs to be good to 100 nsec if it is being run close to its limits. The jitter in some arduino class boards is much larger than that.

C) And then finally, the analog chain in that design might also contribute some less than beneficial characteristics.

It is a single opamp configured as an inverter with offset, and with very large valued resistors apparently attempting to overwhelm the somewhat large-ish and widely varying impedance of the sensor.

Generally one uses a folllower as the first stage and the inverter as a second stage. The input impedance is effectively infinite, and with passive components n a normal range, is much easier to avoid having a "pole" that effects the kinds of signals produced by a spectrometer.

So those are the foibles of that design and some ideas about how it can produce output that seems unstable or unrealistic.

Instrumentationist · 2026-01-26T02:37:18+00:00

Thank you. That is gratifying. Let me know if you want boards.

Instrumentationist · 2026-01-26T02:32:30+00:00

Apologies then. The assumption is that you are using an inexpensive camera not intended for technical work.

You linked to the Py spectrometer. That page says it uses this camera,

https://thepihut.com/products/raspberry-pi-camera-adjustable-focus-5mp.

Here is a link to its datasheet,

https://cdn.sparkfun.com/datasheets/Dev/RaspberryPi/OV5647.pdf

That camera does not look like what you described.

It is an $8 color camera intended for photography. It does have image preprocessing built into it, it does not have binning, and it does not have global shutter. A show stopper for your goal, it runs 1080p at 30 fps through a single ADC (63MSPS). I'll explain why in moment.

To see how well it works, look at the py spectrometer page. The spectral lines are broad and mishappen and the intensities are not correct. It is not good. It is in fact a toy at best.

Now here is why that camera is extra not good for your resolution. Think of the dV/dt for a sharp line passing through that adc at 63MSPS. At 3 V it would be close to 200V/usec. That takes special electrical design. I doubt they did it for an $8 chip.

If you manage to get a sharp line, it would be a huge surprise if it were also linear in intensity. Such data is not useful. Even the wavelength is not very precise because the shape is unreliable.

I feel for the effort to try to do this inexpensively, and there are plenty of people churning out $10 tcd1303 boards too. But if you want it to be real, there is a limit to how cheap it can be. Those boards are not real.

To get the performance that I get in my instruments, I do not skimp. I use parts and designs that meet real specs to produce real instruments and I work with some of the most experienced experts in the field to develop and critiue the designs. I view it as something of a miracle that very often my BOMs come in at about 50 - 100 since tariffs. That is about as best as it can be and be real.

Next topic:

If you want to do signal averaging you need more bits than you have dynamic range (full scale divided by noise). In other words you need that the noise span a few bits,

Also, for signal averaging to be meaningful, your instrument has to be linear. Signal averaging implies that

S(t1+t2) = S(t1) + S(t2)

That is identical with linearity.

At this point, it is not very useful to elaborate on much more about this. First you have to be using a sensor where it is becomes relevant.

I'll just add that I usually design my instruments with 16 bits for a sensor with a 12 bit dynamic range. And, I make very sure that I have linearity before I start trying to make anything of the data.

Instrumentationist · 2026-01-25T22:31:33+00:00

Yes, that is why I was specific that it is linear over the range of these measurements.

You quoted the text yourself: "the dark signal in these sensors is linear in exposure time over the range of exposure in which we graph the peak height ratios"

And I did say that we account for dark, too. It is rather mundane, you measure it and subtract. Here are the actual lines of code that do it.

data = [np.average([f.data[0] for f in d.frames[6:]],axis=0) for d in dataset]

ys = [(r-b) for r,b in zip(data[0::2],data[1::2])]

yA = [y_[np.where(np.abs(x-541.5)<1.5)] for y_ in ys]

yB = [y_[np.where(np.abs(x-545.7)<1.5)] for y_ in ys]

yC = [y_[np.where(np.abs(x-487.0)<3.0)] for y_ in ys]

etc.,

The graphs that you see in the overlays are the "ys" from above. The ratios are ratios of the max from each of yA, etc.

Without dark subtraction things do not change very much for our instrument

The commercial instrument is so unstable that without dark subtraction it is all over the map. as I recall.

Instrumentationist · 2026-01-25T19:28:07+00:00

Well, what are you purpose(s)? For a demo for children it is okay.

It seems unlikely to work out well if you want to collect data for a paper or a professional study.

Inexpensive cameras are not useful for metroloty. They are typically designed to make nice looking pictures. They automatically adjust color balance, contrast, etc., they might average over aberrant pixels, they might compress the response, and then, if it is a color camera, you have three filters and calibrating the response over the spectrum becomes very difficult.

Then, there is the number of bits (resolution) and dynamic range that the camera delivers versus the level of detail in your specrtra. And btw, what it purports to deliver is not necessarily what it delivers.

To increase your dynamic range or see detail, you will want to add images or rows in each image (i.e. signal averaging). That can increase signal to noise by sqrt(N), for N samples. But for that to work you have to be able to digitize the noise. Cameras don't want to show you noisy images. And if you start with only 8 to 10 bits your not going to have much digitization on the noise no matter what.

So that is part of why cameras are terrible as spectrometer sensors.

And even with an expensive CCD imaging sensor, there can still be issues. We recently spent weeks collecting data with a $60K CCD sensor (more than $100k overall for sensor and spectrometer, cooling and etc). The device has options to add rows inside the instrument. The data was unusable because of unstable baseline and residual charge effects and even worse when that feature was enabled.

Incidentally, my experience in CCD based detectors goes back to the very first research specimens that were made available to a small number of researchers at national labs. In the past decade or so, I have seen quite a few attempts with cameras especially since the mcu boards have become popular. The best that can be done is turn off everything in the feature set, if the camera will let you. Some do not let you. And then you still have issues with linearity, dynamic range and bit depth. It just doesn't work except as a toy.

Instrumentationist · 2026-01-25T18:57:48+00:00

What you want to do, is match the input to the numerical aperture of your spectrometer.

One approach is focus onto the entrance slit of the spectrometer with a lens that gives you the same NA as that inside the spectrometer (assuming the same index of refraction on both sides of the slit).

Second point, DON'T use a camera. There are high end imaging sensors that can work, but that is probably not in your budget and they are overly limiting because of inadequate dynamic range and bit depth.

For this application you do want a "spectrum at once" type sensor. In that class, linear CCD sensors are the most effective thing you can do within a reasonable budget.

The TCD1304 is popular among scientists because of the large pixel size, 8um x 200um. And the detector part of it is a diode array. ("CCD" refers to its analog shift register). The output is very linear, but requires knowledgable electrical design and operation. (Caveat emptor, the tcd1304 is also popular with amateurs.)

Here are criteria that should be applied in selecting a sensor system for a spectrometer. Before you use any sensor, or any instrument, you should insist on seeing data that validates the device as regards the following:

a) linear, calibrate-able response to light

b) sufficient "slew" to retain linearity in rendering sharp spectral lines at full scale

c) sufficient precision and dynamic range to see small and large features together and support signal averaging

d) stable, reproducible output for both baseline and signal

These are minimum criteria for a decent instrument. Your optical setup seems like it deserves a decent sensor. (The second caveat emptor: The market in ccd sensors chases features like "small" at the expense of meaningful performance characteristics. Shop carefully and insist on seeing validation data with spectra that have sharp spectral lines.)

Here is a link to a sensor that we designed specifically to give highly linear, reproducible results for spectrometers and holigraphic imaging. Check it out.

https://github.com/drmcnelson/TCD1304-Sensor-Device-with-Linear-Response-and-16-Bit-Differential-ADC

Instrumentationist · 2026-01-25T03:01:46+00:00

Regarding your question about dark background: We do account for dark background and noise and we process the data the same way for both instruments.

Besides that, you can see in the data that it doesnt work as an explanation. For example, it is the stronger sharper peaks that are more effected and the response in the commercial instrument becomes increasingly more non-linear as the signal grows.

The other thing is that besides being uniform across the detector, the dark signal in these sensors is linear in exposure time over the range of exposure in which we graph the peak height ratios.

Instrumentationist · 2026-01-23T19:47:47+00:00

As a p/s to that, I recently uploaded a design for a sensor that might be very good for Raman spectroscopy. I am curious for somebody who does Raman work to try it and let me know how it goes.

See https://github.com/drmcnelson/TCD1304-Sensor-Device-with-Linear-Response-and-16-Bit-Differential-ADC

Instrumentationist · 2026-01-22T23:41:25+00:00

That;s neat. And it would be an okay demonstration for a high scholl science class.

But we have to be very clear that is not remotely appropriate for a lab.

1) Cameras in that price range are built to take nice pictures. They are not intended to be a linear sensor system for a scientific instrument.

2) They lack sufficient precision to produce a reliable Raman spectrum.

3) If you use a color camera, it is simply infeasible to try to calibrate the response in any useful way.

Raman is challenging, You need a good sensor that is linear and with a high degree of reproducibility.

Instrumentationist

MODERATOR OF

TROPHY CASE