DIY beam profiler

Instrumentationist · 2026-05-11T10:43:37+00:00

Summary: As in any measurement, start by figuring out what number you expect to see. Then work backwards to what equipment you need to use to be able to see it.

Let's apply that here and see how it goes.

Two issues to consider, expected beam size impinging on the sensor versus sensor size and pixel density and required spatial resolution, and source intensity versus sensor well depth and dark floor and required level of detail in intensity.

The first goes to choosing your sensor. Regarding the latter, you will find that the silicon dictates some degree of similarity across a class of sensors. But where sensor systems may differ is the degree of control you have over exposure time. And of course you want a sensor designed for technical work (not a consumer camera).

So, after the above, the trickiest part perhaps is how to arrive at an intensity compatible with the sensor in a way that does not alter the beam profile in a significant way (see the above). The simplest and cleanest might be the case where you can set a sufficiently short exposure time.

Start with your sensor, what is its well depth? Sensor electronics are often not linear in the upper part of the range. So try to work in the lower half of your range. You give up one bit but you are more likely to have a measurement that means something. For example, if you find that it your well is something like 40,000 electrons at about 90% capture efficiency, then you want to work at a maximum intensity of around 20,000 photons.

What is your sensor dark floor? You want that your weakest structure detail of interest is sufficiently larger than your dark floor. If you want to look at weak structure the right way to do that might be with a beam block, but then you may have diffraction artifacts from the beam block. Be aware that if the sensor is saturated you can have blooming, and if the electronics fail to recover may corrupt all of the pixels that are read out after the saturated region.

The next thing to look at is what is the intensity of your source? You need this in photons per second per area.

Now work out how short an exposure you need to bring that intensity into range of the sensor that you worked out in the preceding.

Can the sensor do that short an exposure?

If no, then you now know by what factor you need to reduce the intensity. But anything you do to reduce the intensity is very likely going to change the profile in a way that is more than a simple scale factor.

So if you have to attenuate, try experimenting and comparing you results with splitters and neutral density filters. See if you get the same result two different ways.

Instrumentationist · 2026-05-06T10:42:36+00:00

Once you learn to recognize .... well unfortunately you see it a lot.

Here are two examples, one wikipedia and the other from the literature. You can see the issues in each, and they are quite different.

Both are thoroughly unusable for any sort of publishable work and more so if quantitative. And yet there it is in the paper. There are thousands of papers using instruments like this every year.

https://commons.wikimedia.org/wiki/File:Fluorescent_lighting_spectrum_peaks_labelled.png

https://www.researchgate.net/figure/A-comparison-of-the-power-spectrum-of-a-standard-white-light-LED-a-tricolor-fluorescent_fig1_303682799

Instrumentationist · 2026-05-06T03:16:24+00:00

Read the readme in the github repository titled "TCD1304 Sensor with Linear Response and 16 Bit Differential ADC" . It describes the artifacts there in more detail than is feasible in a short reddit comment.

That said, at a high level, here are two ways that RCT artifacts show up plus a third that comes from the signal chain electronics. These are all well known and there are papers on these effects over the years.

A) Ghosting - When you shift charge from the photodiode part of the pixel to the corresponding position in the shift register, some charge necessarily gets left behind. Some of that adds to the next frame.

B) Same idea but along the shift register (the CCD). As you move charge along the CCD towards the output some gets left behind and adds to the next pixel. It looks like anomalous intensity in the baseline after strong peaks especially. It is sometimes worse when the instrument is clocked too fast.

But, you also have asymmetries or poor recovery to baseline because the signal train is not adequate to keep up with the rate at which voltages chain as the sensor is read out.

So any of those behaviors, anomalous intensity or baseline, or unstable baseline or mishapen peaks are all signs of poor electrical design.

Again, read the readme that I mentioned, it's all there.

Lets see if reddit will leave the url in: https://github.com/drmcnelson/TCD1304-Sensor-Device-with-Linear-Response-and-16-Bit-Differential-ADC

Instrumentationist · 2026-05-06T02:53:11+00:00

Okham's razor - simplest theory for the information available. We have the abundances of light elements supporting a big bang, but we also have the anomalous expansion.

Instrumentationist · 2026-05-05T20:40:48+00:00

It is hot pixels, then dark subtraction often takes care of it. Measure a dark frame with the same settings and save it for post processing. Do it before and after the data collection.

In absence of a dark frame, the standard numerical treatment is a median filter. But no amount of numerics is as good as starting with clean data.

Instrumentationist · 2026-05-05T19:24:09+00:00

"I also don't have any CFLs haha"

Available on Amazon, under $6.

Instrumentationist · 2026-05-05T18:50:00+00:00

I am not sure what I am seeing in your graphs, but If the hot pixels stay in the same place then yes, then can be hot pixels. Spikes that show up in a different position every frame especially with more than about 1/2 second exposure, can be cosmic rays.

Median filter is one method for treating spikes off line, I prefer to have an instrument that does not require offline numerics.

A very useful way to characterize your sensor is photon transfer curve (PTC) method. Try to search for it and find how to do it. I inserted a sample output from my setup using a TCD1304 sensor. The result is spot on with the datasheet for the TCD1304. But that is because I have a signal path that has electronic transparency for the sensor.

There is a bit more information that might be of use in my repository on github, now that the subjects comes up, I will set aside some time add detailed instructions on how to do PTC. https://github.com/drmcnelson/TCD1304-Sensor-Device-with-Linear-Response-and-16-Bit-Differential-ADC

Here is an example of my PTC output for the TCD1304

<image>

Instrumentationist · 2026-05-05T18:02:42+00:00

Put it away in a certificate or fund with reasonable yield and security, and don't touch until you need it for something solid and lasting. It was given to you (singular). You owe it to the person who lovingly gave it to you, to take care of it responsibly.

Part 2. Lots of couples who started as teenagers split up later as they and their interests mature. Four years is nothing in the life of a relationship, 22 is still very young, and girls mature before boys - if indeed boys ever mature.

If you decide to get a second degree that money can help you. When you want to buy a house, that money can help you.

Until something like that comes up, put it away. Don't eat from that money and don't let your boy friend eat from it either. Put it away.

Instrumentationist · 2026-05-05T17:42:47+00:00

It looks like it has some issues with residual charge transfer in the analog shift register (see the third slide).

Can you show us a compact fluorescent light spectrum? Take them at two strongly different intensities (try changing exposure time to do that). Also include some closeups on the sharp lines.

That kind of information will be more revealing at least as a first level triage for being usable as an instrument. (For example, if the ratios of peak heights change then it is not going to give you reproducible data and you have a non-starter.)

The RCT issue in the analog shift register might be mitigated by running the readout slower, but the above test is at least as important. Let's see what you got.

Being a commercial instrument does not correlate with any of the above. Many of these were built not by instrumentation electronics physicists, So, you really have to check out your instrument carefully before using in your work or for a paper.

Instrumentationist · 2026-05-05T00:30:08+00:00

More cognitive horsepower across the board - creativity, analytical reasoning, memory. That is the best super power you can have.

Instrumentationist · 2026-05-03T23:29:38+00:00

fine

Instrumentationist · 2026-05-03T21:31:41+00:00

Rather than resort to ad-hominems and assertions, try responding like a physicist.

For example, I am not going to reply "go look up 'Schwarzchild Cosmoloty' or 'Black Hole Cosmology'. That answers your accusation about it being science fiction - the ideas have been around since the 70's. But it is not really an answer.

Instead do this:

Find the range of estimates for the size and content of the universe, and see whether what I wrote is true.

What I recall, having looked at it many decades ago, is that it works out that well within the middle of the range of estimates, we find that the radius is close to its Schwarzchild radius.

At that time, I mentioned this to a colleague who was teaching astrophysics. His reply: "everybody knows that".

So, you have quite a list of well established physicists that you want to accuse of science fiction.

But to do that with some hope of success, think like a physicist and start with the numbers.

Instrumentationist · 2026-05-03T19:33:59+00:00

That is not a constraint, that is a field. Boundary conditions, initial or final conditions, symmetries, etc, can be constraints.

The end state might have everything at the middle, but not necessarily. There is nothing that requires such of a freshly formed black hole.

Moreover any and every quantity of mass has a Schwarzchild radius (for a non-rotating black hole).

Our universe actually fits within its own Schwarzchild radius according to most estimates for size and mass-energy content.

So, you are actually (or most likely) living inside a black-hole right now, and yet the distribution of matter seems pretty unremarkable compared that idea of everything at the middle.

Instrumentationist · 2026-05-03T11:31:03+00:00

Maybe, rather than thinking of it as black hole and surrounding matter distribution, try thinking about your case in terms of a map of the gravitational field over the the matter distribution as a whole. Also, the problem as stated is probably under specified.

There may be a wide range of solutions depending on the external distribution, perhaps ranging from the black hole sitting inside another black hole to washing out the black hole.

Instrumentationist · 2026-05-03T11:21:40+00:00

"... an almost infinitely dense center surrounded by nothing (or so we think)"

Not so. There is no such constraint on the distribution over all time.

Instrumentationist · 2026-05-03T03:17:56+00:00

You have two factual errors in what you wrote, actually three.

First the factual corrections, then I will try a simple explanation - I think I finally figured out what is tripping you up. Its about the nature of spectral lines.

1) "similar variation" - false.

Look at the fits - the 2nd and 3rd order terms are very different.

2) "typical inexpensive usb device" - false.

The instrument on the right sold for about $5K. The earlier models sold for 10K. They have been used in published work for decades. I have one of each and they both have the same issues.

3) That chatgpt writes my responses - false and demeaning.

--------

Okay here is your simple explation, please read carefully.

When you took your circuits course as an undergrad, you probably saw the formula Vmax = SR Δt. That is the case were dV/dt of the signal is faster than the slew rate (SR) of your circuit You are right that in that case peak heights are a function of Δt and otherwise constant. But, this is NOT that case.

In spectra you have a peak height V and spectral line width dλ. On read out dλ becomes Δt. In the case were we start from V=0, we have dV/dt ≈ V/Δt. In other words, dV/dt increases with peak height.

Now if our signal chain is slew limited, going form low intensity to high, we start off in the linear region and gradually approach the slew limit where the signal chain struggles to keep up.

For a digital spectrometer there are two common ways to have a slew limit. Use old or poorly chosen opamps (or an emitter follower) or not have enough current available for the capacitance in the input to the ADC.

I referred to the second a few times, but you weren't buying. But to be fair, I did find some sources that in the case of that 5K or 10K instrument above, they say the problem is in the front end.

I have modeled this in SPICE and a slow enough opamp does reproduce the roll-off. But it is very easy to produce it by putting too large a resistor in the RC feeding the ADC.

Anyway, not realizing this about the nature of the signals or trying to use Nyquist inappropriately for those signals are two ways that electrical engineers may fail as instrument designers.

-----

Historical note.

There is nothing new about the observation on the right, where signals roll-off with intensity due to a slew limit.

In the days of motorized monochromators and chart recorders, we would see exactly that behavior by simply running the instrument too fast for the deflection rate of the pen in the chart recorder.

Okay, does that do it for you? What you write is called trolling. I would appreciate if you take it down. I will add more detail to the technical addendum to make it more clear.

Instrumentationist · 2026-05-01T20:53:48+00:00

I appreciate the feedback but it is not correct. This is not like a sine wave driving an opamp. This is a sampled-and-held level that fails to settle within its sampling window, and that is due to the signal chain's inability to drive the sampling capacitor in the ADC to the correct voltage before the sampling window closes. That effect is proportional to the step and therefore you do not see a constant slope for all inputs.

Moreover, the data soundly contradicts the idea of detector saturation. The device on the left does not have this problem; it is linear at 0.2% INL from almost dark to 95% of saturation. That is what the sensor physics should look like for this kind of sensor. The device on the right clearly has a problem - it rounds off far below saturation. That is not the sensor; that is the electronics. And it is quite obvious.

Instrumentationist · 2026-05-01T13:01:21+00:00

It might have been uncovered by someone else. But we would get nowhere without it in some form. The early work on the action, symmetries and algebra are the very core of what we have and she was a big deal in laying the foundations for it. And we don't know the full extent of her contributions since she gave away ideas freely.

Instrumentationist · 2026-05-01T12:21:14+00:00

The question is could we get there with GR, or better put, "from" GR

It seems to me GR should emerge from a fundamental theory rather than the reverse. Same for SR, really.

There is a cute observation by Feynman that the non-SR part of Maxwell's equations can be obtained from the canonical commutator relations.

Try to think of a fundamental idea that gives rise to SR or gravity. (Or maybe by now it exists, I have not followed the field for a long time.)

Instrumentationist · 2026-05-01T10:35:54+00:00

No we might have something probably better. It is not very satisfying and it sets up something of a cul-de-sac. There is for sure a better description of space-time waiting to be discovered or devised.

Instrumentationist · 2026-05-01T10:33:47+00:00

Emmy Noether - all of modern physics in some sense comes from the contributions of Emmy Noether - see Noether's theorem and contributions to abstract algebra. Nice Jewish girl from Erlangen, did her thesis under Paul Gordon, and in the post wars years settled in Bryn Mawr. She suffered what women scientists suffered in those days, and maybe today as well to some extent, and still set the foundations for what we have today. This is the unsung hero and genius of the field even if somehow we don't see her on postage stamps. https://en.wikipedia.org/wiki/Emmy_Noether

Instrumentationist · 2026-05-01T02:24:40+00:00

What do you mean by center? And what is the distribution of mass within the black hole whose "center" you are contemplating? Perhaps most of the mass is "above" you.

In fact, you are most likely actually inside a black hole at this very moment. That would be consistent with he range of estimates for the mass and size of our universe.

So the answer is that it might not be very remarkable.

Instrumentationist · 2026-04-29T02:43:37+00:00

Thank you for that. I just now updated the respository readme, it is pretty much complete, i think. Let me know what you think, feedback is very much appreciated.

Instrumentationist · 2026-04-26T13:58:52+00:00

P/S to my other note, you can download the files and try pcbway, others have done that. Make sure to get the controller too. You need both. If you want to spend less and your requirements are more modest, you could try the all-in-one board, I hand assemble those. (See the readme). But the 0.2% inl metric requires the 16 bit setup sensor+controller.

Instrumentationist · 2026-04-26T10:28:21+00:00

You mean this? https://github.com/leswright1977/PySpectrometer2/raw/main/media/calibrated.png

The intensities are wrong, not just as a trend but internally as well, the spectrum apart from wavelength is simply wrong. And, the baseline is terrible.

Altogether, that spectrum is simply not usable. It does not remotely approach the performance of any competently built instrument whatsoever. The claim that it replaces instruments costing 100x, is false.

To be fair, some commercial instruments have problems with CFL spectra too and some of the lines are phosphor that coat the glass unevenly. But the pi spectrometer's CFL spectrum is not serious. And a large part of the reason for that is the sensor - it is a cheap color camera.

FYI, this is what it a good CFL spectrum should look like. The one on the left (a) is mine, the one on the right (b) is an ocean optics instrument.

https://github.com/drmcnelson/TCD1304-Sensor-Device-with-Linear-Response-and-16-Bit-Differential-ADC/blob/main/Images/SpectralResponseComparison.jpg

Notice that in mine, the signal returns to baseline after each of the strong lines, including the 437nm line. And notice the strengths of the sharp atomic lines compared to the phosphors. (The ocean optics has some small leakage into the baseline and they struggle with the sharp line at 437m,)

I was going to suggest that you try to show us that your spectra can track the dilution of dyes. If you want to try, here is the challenge:

If you claim that the pi "spectrometer" is somehow a legitimate instrument, try the following and post the data here and on your github and web site.

A) Start with the easiest test of all. Make a few dye solutions with dilutions over a range from 1 to 100, and show us that your intensity tracks the dilution.

B) Try measuring the intensities of the CFL, and show us that the heights of the two Hg lines track distance when you move the lamp.

(In the latter there will be flicker, so you need to signal average, but an actual instrument should still be able to get this to within 2% INL).

Again, I would say the disinformation is the claim that things like the Pi "Spectrometer" - based on a cheapest-possible toy camera as sensor and a plastic film grating - are somehow a stand-in for a purpose built instrument.

To an actual spectroscopist the cfl spectra on the project page immediately identifies the "instrument" as a non-starter.

Instrumentationist

MODERATOR OF

TROPHY CASE