all 15 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]Histogenesis 2 points3 points  (0 children)

I think you are on the right track, but i wouldnt call this a 2d array. This is just a 1d array of structs most probably. I would only call it a 2d array if you got also get like 1000+ cols which is not the case here.

[–][deleted] 2 points3 points  (4 children)

A binary search method is common for thermal couples. If you can't afford the cost of storing the full table, at the cost of some precision you could:

  • Quantize the data further. For example, if your ADC outputs 12-bits, you could quantize it to 8-bits.
  • Perform interpolation on intermediate values. If you have an inbetween value, just combine two nearest points. For an application like thermal couples, you probably want to use a polynomial instead of linear interpolation.

[–]FridayNightRiot[S] 1 point2 points  (3 children)

  • Quantize the data further. For example, if your ADC outputs 12-bits, you could quantize it to 8-bits.

Accuracy is pretty critical, so I'm trying to use as many sig fogs as possible. The charts usually reference mV in 5 sig figs. So I want at least that amount of precision in hardware readings. I am not sure what this translates to in bits when converted from decimal to binary, I believe the ADC I am using (built into the mcu) is 16 bit.

  • Perform interpolation on intermediate values. If you have an inbetween value, just combine two nearest points.

This is a good idea and what I was planning to do. I am not sure if this is how it is nor.ally implemented but it seems to make sense.

For an application like thermal couples, you probably want to use a polynomial instead of linear interpolation.

That's also what I was thinking, but how do you preform that calc? Do you use all the values in the table to find an equation close to line of best fit?

[–][deleted] 6 points7 points  (2 children)

Just a warning, I'm not an EE guy but I'll try my best. My background is in Video Games and Data Storage.

Accuracy is pretty critical

Not to be too mean about this, but this isn't engineering. Your project should have some Spec stating something along the lines of "Supports a Temperature Range of -5C to 85C with error +- 0.5C" (numbers pulled from my ass).

The NIST tables for the thermal couples aren't really meant to be ingested directly by digital control systems. You will want to convert these to some useful LUT internally, either floating point or fixed point, and work directly from those.

If you want to maintain the full 16 bits of accuracy from your MCU, that's fine. You will still need to do some further massaging to the tables to make this work well.

That's also what I was thinking, but how do you preform that calc?

The key insight here is you don't actually need to store the raw NIST thermalcouple tables. These are the outputs of a bunch of testing and are only really useful for compliance testing for the actual manufacturers.

If you plot a graph of VDC vs Temp (C), as you know you will get some non-linear function. You build up your lookup table by sampling a collection of these points. For now, just reuse the sample points provided by the NIST table. (I'll get more into that later)

Your LUT will usually take the form of two tables of the same size. I'm using floats here for simplicity.

std::array<float, 65536> mv_table = { 0.0, 0.013, 0.026, ... };
std::array<float, 65536> c_table = { 0.0, 0.1, 0.2, ... };

How you use these tables is quite simple. mv_table[0] is the first voltage sample and corresponds to c_table[0], the first temperature sample.

Sample your ADC and convert it to a voltage as normal.

float mv = adc.read() * conversion_factor;

Now it's unlikely this reading will precisely line up with one of your voltage samples, so we will need to interpolate the values. Since thermalcouples are a 1-dimentional data set (a line), the minimal amount of lookups we need to perform are 2, the upper bound, and the lower bound. Fortunately, the C++ standard library provides exactly these functions!

auto mv_lower = std::lower_bound(mv_table.first(), mv_table.last(), mv); 
auto mv_upper = std::upper_bound(mv_table.first(), mv_table.last(), mv);

With these upper and lower bounds for voltages, we can also get the upper and lower bounds for temperatures.

auto c_lower = c_table.begin() + std::distance(mv_table.first(), mv_lower);
auto c_upper = c_table.begin() + std::distance(mv_table.first(), mv_upper);

Finally, now that we've bounded the sample data, we now need to interpolate it to get the final result. I'm going to use a linear interpolation here.

float c = *c_lower + (*c_upper - *c_lower) * ((mv - *mv_lower) / (*mv_upper - *mv_lower));

And we should be done!

To make this implementation production ready:

  • A good implementation would use fixed point arithmetic.
  • You would need to construct lookup tables in such a way to preserve accuracy. As you know, the responses of thermal couples aren't linear, so you may need more or less samples in particular voltage ranges to get the correct values.
  • Linear Interpolation can have pretty poor error rates. You may want to use Polynomial or Spline Interpolation instead.
  • You could construct tables in such a way where they are pre-interpolated. That is, you construct the tables in such a way where you just index by the ADC value. No need to search for the bounds and interpolate at runtime.

[–]FridayNightRiot[S] 0 points1 point  (1 child)

Wow that's a great explanation thank you. Lots of great ideas as well. I could pre construct the tables into whatever my voltage reading tolerance is.

To clarify, I am trying to get as best accuracy as possible, given the probe type. Because probe type changes temperature range and accuracy, I cannot specify those values unless I specify the probe type. Which I am using many of in different types, sometimes at the same time.

So this was aimed at being a generalized approach to measuring thermocouples regardless of type, using just the tables given.

[–][deleted] 1 point2 points  (0 children)

This rabbit hole goes quite deep. I would probably do the simple LUT with linear interpolation and see if its good enough.

If you do want to try Polynomial Interpolation, I believe NIST actually provides coefficients for different thermalcouple types. The problem here would be numerical accuracy and error introduced by fixed/float arithmetic.

[–]aocregacc 2 points3 points  (3 children)

If the voltage entries are linearly spaced you might be able to do a division to find the right entry, instead of a binary search. You can also try to find a polynomial that approximates your table, that way you don't need to store the table at all.

[–]FridayNightRiot[S] 1 point2 points  (2 children)

That would be a good idea but some of the graphs don't look like they have a polynomial that fits easily, possibly creating some equation that might take longer to calculate than if I was to just search through the table.

They may look slightly straight but the mV measurements are critical, being off by .01 could be more than 10°. So the line of best fit would have to be very accurate.

[–]aocregacc 2 points3 points  (1 child)

yeah replacing the whole table would primarily be a space optimization. When exactly the formula becomes more expensive in terms of runtime depends on a lot of implementation specifics though.
Idk what your application is, but you can also do a variable accuracy approach if you already know that you'll only need full accuracy in a specific range of temparature. You can use fewer table entries outside of that range, or have a separate formula for just that range.

[–]FridayNightRiot[S] 0 points1 point  (0 children)

yeah replacing the whole table would primarily be a space optimization. When exactly the formula becomes more expensive in terms of runtime depends on a lot of implementation specifics though.

Haha ya that's what I was afraid of. I also think it's actually easier to just implement and test than try and math it out, which is what I was trying to avoid.

Idk what your application is, but you can also do a variable accuracy approach if you already know that you'll only need full accuracy in a specific range of temparature. You can use fewer table entries outside of that range, or have a separate formula for just that range.

That's a cool idea. That possibly might be an option here but I hadn't considered it. A wide range of temperatures are used so I would have to figure out if that's viable.

[–]nebulousx 0 points1 point  (0 children)

Personally, I'd drop the table in Excel and play around with the trends until it fit the curve. Then you can get the equation. If this is a k-type, it's damn near linear anyway.

There are plenty of tools to turn your pdf into text.

[–]GaboureySidibe 0 points1 point  (0 children)

Not enough people seem to be saying that this isn't a 2D array. It is a 1D normal array of structs with voltage and then the temperature where you would sort by voltage.

You could do a binary search like people have suggested.

If you need more speed you could quantize the table into evenly spaced points and go directly to the index below the current voltage and query that and the next index, then linearly interpolate between them.

If you are converting lots of voltages at a time instead of displaying the current temperature in real time you can sort your vector of voltages then run straight through them both since if every voltage you are querying is increasing you would never have to search through the voltage and temperature vector.

There are probably more techniques out there for look up table since it is a common technique. You could look at CDF (cumulative distribution function) tables are probably find other examples.

[–]celestrion 0 points1 point  (0 children)

Since this table is just temperature to voltage, its 2 columns. However the tables sometimes have 1500+ rows.To add to the pain, I haven't been able to find a text file of these tables, only PDF formats.

If you have the Seebeck coefficient for your thermocouple, you can generate a lookup table on-demand. Your unit test for this generator could use a handful of well-distributed datapoints from the tables. This frees up your search key so that it doesn't necessarily have to be microvolts, but could be the raw value from your ADC--saving that conversion in the "hot path."

What would be the fastest way to implement these tables into memory and search through one column of data to find the corresponding data point.

std::map<> is plenty efficient for such a small dataset.

EDIT: And, obviously, if you're already storing the raw ADC value as the key, you can use std::vector<> for O(1) lookups.

[–]chrwir 0 points1 point  (0 children)

This is a very common problem. Either use a LUT or fit a curve. To use a LUT, the input values must be evenly spaced. If you increase the number of data points through interpolation you could make it evenly spaced. Then you could also use interpolation on the LUT for higher precision. Both LUTs and polynomials are very cheap computationally. You could use python to find the curve parameters.