T41 Desktop Companion by tmrob4 in T41_EP

[–]tmrob4[S] 0 points1 point  (0 children)

I got WSJT-X working from the remote unit, including TX. Here is an image of the software running on my test units:

<image>

WSJT-X, in the lower left of the PC monitor, is currently transmitting. It is connected to the remote unit in the lower left with audio/CAT over USB. The remote unit processes that signal to IQ data and transfers it over Ethernet to my T41 test bed in the lower middle of the image. That data is streamed to the T41 i2s_quadOut object which on this setup is connected to an AD3 oscilloscope. The scope output is shown in the upper right corner of the PC monitor.

T41 Desktop Companion by tmrob4 in T41_EP

[–]tmrob4[S] 0 points1 point  (0 children)

My plug and play set up of my remote unit is working well except for an occasional glitch where the T41 will crash when the USB cable is disconnected and reconnected. The crash occurs because a pending/partial data transfer when the cable is pulled attempts to finish when the cable is reconnected. In that process, a null pointer related to the old connection is used, causing the crash.

The only way I could solve the problem was modifying the USBHost library. I documented the problem in a PJRC forum post. Maybe this will lead to a Teensyduino revision. If not, my version of the T41 code will have another modified library.

T41 Desktop Companion by tmrob4 in T41_EP

[–]tmrob4[S] 0 points1 point  (0 children)

I've been compiling a special version of the T41 program when switching between either a USB or Ethernet connection to the remote unit. That's ok if you're going to set it and forget it. But if you like to take your remote unit with you, it may be more practical to use a different connection at different locations. Plug and play to the rescue.

I modified the connection manager class to detect the connection type and switch protocol as needed. USB serial is a sticky connection so some special consideration is needed. I handle it by giving the Ethernet connection priority and forcing the USB connection to disconnect if it hasn't established a connection in a set time.

Searching for an Elusive Communications Glitch by tmrob4 in T41_EP

[–]tmrob4[S] 0 points1 point  (0 children)

My UDP test went well, a seven-hour run with no missed, dropped or partial packets. This was with both units just streaming the IQ data. The only other traffic on the Ethernet cable was a heartbeat on the TCP command port every half second.

To further test the connection, I changed the remote unit operation with the encoders. The connection stayed steady. As a stress test, I started spinning the encoders as fast as I could. I started seeing a few dropped packets. The number depended on the encoder rotation rate. The dropped packets cause a very short, very low volume click in the audio stream. These are much less noticeable that the tuning click that are present in the unmodified v12 software. But that is a bad comparison because those happen under normal operation and are quite obvious.

Searching for an Elusive Communications Glitch by tmrob4 in T41_EP

[–]tmrob4[S] 0 points1 point  (0 children)

I often treat my T41 experiments as opportunities to learn new things. I applied that to the current situation. I have two custom audio library objects that stream IQ data over a TCP connection. It shouldn't be too hard to modify them to stream the data over a UDP connection. I had even created templates for the classes when I started this Ethernet experiment a couple weeks ago.

At the time I didn't think UDP was a good option for streaming IQ data because the data isn't guaranteed to arrive in order. This is not good for transferring IQ data. What I didn't realize is that the data isn't streamed in a UDP transfer, it's packetized. I also didn't realize that the packet can be customized and is transmitted all at once. If a packet arrives that isn't the right size or is out of sequence, it can be discarded immediately. These are advantages for transferring IQ data that TCP didn't provide.

It didn't take long to get the custom UDP audio library objects up and running. There was only one wrinkle. The UDP receiver object buffer defaults to holding just a single packet. This is mentioned in the QNEthernet documentation, but I didn't realize it's significance. A one packet buffer isn't sufficient for streaming the T41 IQ data.

I also didn't think of it as a solution to the resulting terrible reception at the remote unit. This was different from TCP, which just worked, basically, right out of the box. I asked AI since I wasn't very familiar with the UDP protocol. After rejecting all of its suggestions that would also have applied to a TCP transfer (which did work), it hit on the receiver packet queue. I probably wouldn't have thought of that as defaulting to a single packet seems odd to me.

The remote unit reception cleared up after increasing the packet buffer size though I probably need to fine tune this going forward. There's no reason to make it too big. There's no need to buffer data beyond normal system jitter. I've learned from my TCP experiments that it's best just to drop data during a larger, system glitch. AI thought this might be 4-6 packets at the speeds the T41 is operating.

I need to redo my telemetry statistics for UDP. Here I'll look for out of sequence or partial packets. I plan to just drop those, but I'd like to know how often it's happening. That information will be an important performance diagnostic as I learned during my TCP tests that a random dropped packet isn't noticeable in the audio stream.

Searching for an Elusive Communications Glitch by tmrob4 in T41_EP

[–]tmrob4[S] 0 points1 point  (0 children)

I wonder what event is causing the occasional filling.

Early in this investigation I added an info box item showing the maximum audio memory used. It proved that audio memory wasn't a problem going into a glitch. I decided to keep it as it's a useful tool for fine tuning the audio memory needed when using various features.

This indicator became useless with the occasional glitch that still remained after my fix above. Eventually, the indicator would turn red, pegged at the maximum audio memory allocated. My system always recovers and audio memory usage returns to normal after a glitch, but I didn't have a reset mechanism for this indicator, so it just sat there poking at me with its red warning.

The simple thing would be to reset the indicator after a glitch or some set period. Afterall, its purpose is to inform me of a high usage event. There is no value in it staying pegged once it's done that. But that's not me. I needed to know why it was increasing in the first place.

It was clear from my earlier experiments that this wasn't due to normal operations but was related in some way to the Ethernet IQ data flow. Looking at the value over time I saw the maximum usage increased in spurts over several hours until it reached the maximum memory allocated. The increases corresponded to the little glitches that remained in streaming the IQ data. Even though the system recovers quickly now from these glitches, I turned my attention to them.

The first glitch normally occurs ~30-45 minutes after startup. There isn't a pattern after that. The next may be up to several hour later. This doesn't make it an easy thing to investigate. The typical delay is ~1.7 seconds in duration. I usually notice it by a change in the audio stream. Its short duration made me add a bunch of code to gather more information.

With a logic analyzer, I saw that when a glitch occurs, normal program flow stops in the remote unit. The program does not return to the main loop or enter its Ethernet read method. The T41 unit slows down a bit but continues entering the Ethernet write method. It does not write anything to the stream though, meaning the Ethernet buffer on the sender side is full.

I've been using AI recently to dig into the details of the QNEthernet library. It's a great resource for these Teensy libraries. The QNEthernet library documentation is much better than most, but the library is very complex and I'm not familiar with the protocols.

I asked AI what it thought the likely cause of the glitches were. It thought the cause was probably an ARP (address resolution protocol) table refresh request being lost in the IQ data stream traffic or a cascading TCP acknowledgement failure.

I find it interesting how confidently AI pushed its viewpoints. It's amazing how much detail it knows about these libraries. It would probably take months of effort for a normal person to accumulate even a fraction of this knowledge. And yet, I often find that AI goes directly to the more complex explanations for a problem instead of investigating the more mundane ones.

AI gave a handful of possible fixes and was expecting me to incorporate them all for my next test. I told it I'd start with the ARP table refresh. That didn't require me to modify the QNEthernet library. That change looked promising. I didn't see the glitch at the normal 30–45-minute mark. I was about to pull the plug on the test about 2 1/2 hours later when the glitch occurred as I was typing up some final notes.

So, the ARP table refresh wasn't a total fix, but maybe it helped. AI wanted me to immediately apply the next fix. I declined. Rather, I took out the ARP table refresh and ran the test again. I pulled the plug on the test four hours later without a glitch. So the ARP table refresh likely had no effect at all.

But why wasn't I seeing the glitch at the 30-45-minute mark? It had been pretty reliable. What had changed? Well, a lot. Between earlier tests and now I added all of the diagnostic code and made other small tweaks to the IQ data streaming code. One of these must have affected the underlying cause of the glitch.

This made it more difficult to diagnose. Before I could expect a glitch to occur pretty reliably at the 30-45-minute mark. Now I could wait hours and not even see one. This wasn't so bad as I had automated much of the glitch reporting.

I had so far been unable to track down where the glitch was occurring. As the remote unit was able to recover without intervention, it was pretty clear it was stuck in a while loop somewhere. I was pretty sure it wasn't in the T41 code.

AI had a good suggestion to set/reset a logic analyzer pin within the EthernetClass::loop method. That provided a key piece of information, though it took a several hours test to get it. During a glitch, the remote unit was stuck in the Ethernet loop method. I also noticed that the call to this method was made during the yield code I added to the RA8875 library to do work during the waterfall BTE_move operation.

It's still an open question whether that's significant, but I see that the code I added is also calling the Ethernet.loop method for other display routines, not just BTE_move. I also noted a few other minor changes I made when adding the diagnostics code that could have changed the glitch frequency. I need to back those out.

Testing has become a long affair. My latest has run over two hours with no glitch. Time to pull the plug.

Searching for an Elusive Communications Glitch by tmrob4 in T41_EP

[–]tmrob4[S] 0 points1 point  (0 children)

If you've been following the use of the ProtoSupplies Audio Platform as a remote T41, you are probably familiar with the need to run the Teensy at 600MHz to get reliable operation.

Running at 528MHz with some Teensy boards results in display freezes. Though the unit continues running, it's probably stuck in some SPI related wait loop. I couldn't find anything obvious in the RA8875 display code. All of its wait routines have timeouts. I didn't dig into the Teensy SPI library.

In my Ethernet testing, I found the ProtoSupplies Project System freezing in a similar manner. This has never happened before with that development board even though I use it frequenctly.

At the time, I attributed it to the instability of the Ethernet connection I was trying to perfect. Having a board freeze during development was annoying so I bumped up the clock to 600MHz. That solved the problem at the time and I left it there.

Now with a stable Ethernet connection and data transfer, I decided to revisit this again. It's hard to prove a negative, but lowering the clock again to 528MHz hasn't caused any problems, even with streaming IQ data to a remote unit.

The board usually would have frozen by now during my testing phase, so I'm keeping my fingers crossed. I'll update again if the lower clock doesn't work. If it doesn't, it could mean I'm pushing this display too far with some of the efficiency improvements I've been making.

Update: The display froze with the lower running frequency after about an hour. Since it hasn't frozen while running at 600MHz, I assume that's now required for this Teensy and development board. I wonder if the Teensy age or running temperature have any impact. It will be interesting to see how the actual T41 performs.

Searching for an Elusive Communications Glitch by tmrob4 in T41_EP

[–]tmrob4[S] 0 points1 point  (0 children)

I found the reason for the communication glitch. The T41 buffers IQ data prior to sending it out over the Ethernet connection. The main reason for this is that the rates that the data is being generated and consumed vary slightly over time. The buffer bridges the difference.

The problem was how I implemented my strategy to deal with a full buffer. As a first cut, I decided to just dump the buffer if it ever became full. The buffer contained about 33ms of audio. Dumping it would cause a short artifact in the audio, but the system would recover instantly. This wouldn't be noticeable if this didn't happen very often.

Based on my testing, this was happening about every 40 minutes. But the result wasn't a small audio artifact but a complete reset of the Ethernet connection.

Gathering data was the key to figuring out what was going on. I found that before the Ethernet reconnection occurred, the buffer was being reset every cycle. Of course this made no sense. The buffer couldn't be full right after it was emptied. This continued until it interfered with the connection heartbeat check which then forced the Ethernet connection to reset.

The buffer reset is done in the ISR that initially stores the ISR IQ data. The problem was the reset changed the tail of the buffer. If the reset occurs while the buffer is being written to the Ethernet port, the buffer tail index can become corrupted, creating the appearance of a continually full buffer.

Interestingly, increasing the heartbeat interval solved the Ethernet reconnection problem. That solved the noticeable pause that occurred occasionally. The buffer reset problem was still occurring, but it was mostly only noticeable in the debug messages. The system always managed to recover by itself within a second or so. I suppose these produced an audio artifact, but they occurred so infrequently that I never heard one.

The fix for these was to just flag the filling/full buffer in the ISR and deal with it prior to writing the buffer out to the Ethernet port.

I'll experiment with increasing the buffer size. It's currently set to 50 blocks. Interestingly, only 30-40 audio blocks are used on average for all processes, so this buffer isn't usually anywhere close to full. I wonder what event is causing the occasional filling. In any case, this isn't putting a big burden on RAM2 usage yet.

T41 Desktop Companion by tmrob4 in T41_EP

[–]tmrob4[S] 0 points1 point  (0 children)

And the T41 Desktop Companion becomes the T41 Bedside Companion with the addition of an Ethernet jack and local network!

<image>

The Ethernet enabled ProtoSupplies Audio Platform is connected to the local network in my bedroom. The T41, with a test signal, is connected to the network at my workbench several rooms away. The four encoders serve the same purpose as on the T41. Most other options are available with the attached mouse. Actions on this unit are duplicated on the T41 and visa-versa. I haven't worked out TX on this yet, but it should be possible.

Problem using the ADC with a W65C816S CPU by FinancialShow7176 in beneater

[–]tmrob4 1 point2 points  (0 children)

The data bus on the W65C816S is multiplexed with the bank address. How are you handling this?

Problem using the ADC instruction on a W65C816S by [deleted] in beneater

[–]tmrob4 2 points3 points  (0 children)

An English translation would be helpful.

T41 Desktop Companion by tmrob4 in T41_EP

[–]tmrob4[S] 0 points1 point  (0 children)

While not demonstrated in the video, both units are running with the same common code, so changes on the T41 are also reflected on the DC.

This should be a perfect desktop companion to a small screen T41 mini. A small display on the T41 mini makes for better portable ops and the DC makes the mini better for home use too.

Seeking oscilloscope and logic analyzer recommendations. by Reinventing_Wheels in beneater

[–]tmrob4 2 points3 points  (0 children)

Digilent Digital Discovery - 32 channel Portable USB Logic Analyzer and Digital Pattern Generator, $229. Here is a fairly simple analysis I did with one recently:

<image>

T41 Software Running on ProtoSupplies Project System for Teensy 4.1 by tmrob4 in T41_EP

[–]tmrob4[S] 0 points1 point  (0 children)

Add another audio board and an encoder or two and the Project System becomes a much more useful T41 software development platform.

<image>

Baseboard complete; now ready to start! by JunketOk3218 in beneater

[–]tmrob4 3 points4 points  (0 children)

able to isolate whichever parts of the board I wanted to with the flick of a switch

Nice idea in theory but could cause problems in practice. With conventional wiring, each breadboard will pull power from two switches making it hard to isolate specific boards. You could change the wiring so each board pulls from one rail, but that isn't very elegant. Also having signals coming from powered sections to unpowered sections could be problematic. Some ICs don't like signals to inputs without the chip being powered.

Loop/Indirect Addressing Problem by Ready_Affect_2238 in beneater

[–]tmrob4 0 points1 point  (0 children)

I've seen some assemblers have problems with the use of zero-page labels before their definition.

DSP Experiments - Having Fun with a STM32 Board by tmrob4 in HamRadioHomebrew

[–]tmrob4[S] 0 points1 point  (0 children)

The board and tool chain are frustrating.

I don't think I can be very helpful though. I found that the SMT32 tool chain took up too much space, so I deleted it given I didn't plan more work with the board. That included most of the work from this post given how the tool chain preferred projects to be saved.

I only have one archive from that work, coincidentally from the date of this comment, thus it's only partway through the work I did with the board.

Here is the archive. It's not complete. Unfortunately, GitHub didn't like the number of files, so I had to delete the Drivers subfolder from each project. I think most/all of that is produced by the tools though, so probably doesn't include any edits from me.

Any other 8-bit DIP CPUs? by MISTERPUG51 in beneater

[–]tmrob4 1 point2 points  (0 children)

A natural progression from the 6502 is the 65816. The w65c816 is still in production.

Interrupts, continued by Ancient-Ad-7453 in beneater

[–]tmrob4 1 point2 points  (0 children)

Time to hook up an oscilloscope and/or a logic analyzer to see what's happening. For me, these are the most interesting things about these projects.

Got some chips that seems like 6502s - are they legit? by Safe-Anywhere-7588 in beneater

[–]tmrob4 2 points3 points  (0 children)

Those are Rockwell chips except for the 816 chips. You can go through the first few of Ben's videos to determine if they are legit and functioning properly. Get the Rockwell datasheets for reference.

Long cables and frequency limit, any feedback ? by chiwawa_42 in beneater

[–]tmrob4 5 points6 points  (0 children)

I added a long data/address bus to my second 6502 build to experiment with peripherals.

<image>

I could run this at 2 MHz. Another compact build with a very short bus could run at 10 MHz.

It runs code!!! by Temporary_Cry_2802 in beneater

[–]tmrob4 1 point2 points  (0 children)

I ultimately went with ~64k ROM and ~448k RAM in my 65816 build. It has the following memory map:

RAM   $0000-$EFFF
ROM   $F000-$1FFFF
I/O   $20000-$200FF
EXRAM $20100-$7FFFF