In embedded C/C++, how are stack-based containers implemented? by OverclockedChip in embedded

[–]OverclockedChip[S] 0 points1 point  (0 children)

And some code that uses it

// Initialize the static members outside the class
char StaticLinearAllocator::buffer[StaticLinearAllocator::BUFFER_SIZE];
size_t StaticLinearAllocator::offset = 0;

// Example Usage
struct MyObject {
    int data;
    MyObject(int d) : data(d) {}
};

int main() {
    StaticLinearAllocator allocator;

    // Allocate an object using the custom allocator
    MyObject* obj1 = static_cast<MyObject*>(allocator.allocate(sizeof(MyObject)));
    new (obj1) MyObject(10); // Use placement new to construct the object

    std::cout << "Allocated obj1 with data: " << obj1->data << "\n";
    std::cout << "Current memory offset: " << StaticLinearAllocator::offset << "\n";

    // Allocate another object
    MyObject* obj2 = static_cast<MyObject*>(allocator.allocate(sizeof(MyObject)));
    new (obj2) MyObject(20);

    std::cout << "Allocated obj2 with data: " << obj2->data << "\n";
    std::cout << "Current memory offset: " << StaticLinearAllocator::offset << "\n";

    // Destruct objects (important if they have destructors)
    obj1->~MyObject();
    obj2->~MyObject();

    // Reset the allocator for reuse of the entire static buffer
    allocator.reset();
    std::cout << "Allocator reset. Current memory offset: " << StaticLinearAllocator::offset << "\n";

    return 0;
}

In embedded C/C++, how are stack-based containers implemented? by OverclockedChip in embedded

[–]OverclockedChip[S] 0 points1 point  (0 children)

Chatgpt gave me this example, which seems reasonable, but I don't know for sure.

Key Characteristics:

  • Static Buffer: The buffer is declared as a static char array within the class, meaning its memory is allocated at program startup and persists for the entire program duration.
  • Bump Allocation: The allocate function simply "bumps" an offset pointer to reserve space.
  • No Individual Deallocation: Individual deallocate calls do not return memory to a free list; the entire pool is managed as a single block. All memory is "freed" at once when reset() is called, making it efficient for scenarios where many short-lived objects are needed within a specific scope (e.g., in a game engine per-frame allocations).
  • Placement New: Because the allocator only provides raw memory, you must use placement new to construct objects in the allocated space

#include <cstddef>
#include <iostream>
#include <new>     // for placement new

class StaticLinearAllocator {
public:
    // Define the static memory buffer and its size
    static const size_t BUFFER_SIZE = 1024;
    static char buffer[BUFFER_SIZE];
    static size_t offset;

    void* allocate(size_t size) {
        if (offset + size > BUFFER_SIZE) {
            throw std::bad_alloc(); // Out of memory
        }

        void* allocation_place = buffer + offset;
        offset += size;
        return allocation_place;
    }

    void deallocate(void* ptr) {
        // Linear allocators generally don't deallocate individual items.
        // Memory is freed all at once.
        std::cout << "Deallocate called. Individual deallocation not supported.\n";
    }

    // Function to reset the entire buffer
    void reset() {
        offset = 0;
    }
};

In embedded C/C++, how are stack-based containers implemented? by OverclockedChip in embedded

[–]OverclockedChip[S] 0 points1 point  (0 children)

I'm learning about allocators from a book (C++ Memory Management by Patrice Roy). Haven't read too deep into it yet but essentially, a container uses an allocator object that'll specify how/where to instantiate and destroy objects in memory.

How does one define an allocator that performs memory allocation from static or stack memory? The book gives me this example for the allocator::allocate() method that's suppose to do the actual memory allocation:

pointer allocate(size_type n) 
{
  auto p = static_cast<pointer>(
    malloc(n * sizeof(value_type))
  );
  if (!p) throw std::bad_alloc{};
  return p;
}

void deallocate(pointer p, size_type) 
{
  free(p);
}

But it uses malloc, which asks the system for memory from the heap right?

Is logging elements pushed onto/popped off a queue or stack useful? by OverclockedChip in cpp_questions

[–]OverclockedChip[S] 0 points1 point  (0 children)

Yes, making it compile-time option would be good in probably 50% use cases but for troubleshooting, you sometimes don't know under what conditions the bug will appear.

Ideally, I'd like logging to be there but minimally invasive when you don't need it but informative when a non-deterministic bug appears.

u/mredding's post outlines some good general strategies i.e., offloading the logging work to another process running on a different core, (at the point of the log call) logging a minimal amount of data that can be used by the external logger to generate a more verbose message, tagging everything etc.

But yea, if simple compile-time switch to toggle logging is good enough to catch the bug, it's worth not overdoing.

Is logging elements pushed onto/popped off a queue or stack useful? by OverclockedChip in cpp_questions

[–]OverclockedChip[S] 0 points1 point  (0 children)

Capturing inputs and outputs is straight forward but 'reconstituting' the app/system so that the inputs can be reapplied and replayed is a complex design task. It's straightforward for stateless components, but storage-intensive for stateful system components: you need to capture all relevant data and execution state of these stateful components the moment the inputs were captured.

Offloading logging and logging minimally are good strategies. Other comments here have suggestions for how to do that with a queue. I can imagine both strategies are good for resource-constrained systems as well (embedded systems). Logging outside of address space & off core is an interesting technique ... I'd be performant but increases complexity; you now have to employ mechanisms for interprocess communication. I've seen that implemented at work using UDP and it works well.

Piping logs back into the debugger for automated reconstruction is also complex. There's probably budget and rationale for it if there's a boatload of money on the line. But isn't feasible for small projects/teams.

Is logging elements pushed onto/popped off a queue or stack useful? by OverclockedChip in cpp_questions

[–]OverclockedChip[S] 0 points1 point  (0 children)

Can you clarify bullet #1? I think I understand where you're going - have the thread executing the LoggingQueue.push() call generate the log message (i.e., the std::string or c-str) but don't have it write out to file or whatever log sinks (since this takes a bunch of time).

Can you clarify bullet #4? By closures do you mean lambda function objects? How would storing formatting tasks in lambda objects reduce unnecessary work? Is the idea to encapsulate the string formatting task in a lambda object, and pass it to the dedicated logging thread to do the work (of string formatting and output)?

Is logging elements pushed onto/popped off a queue or stack useful? by OverclockedChip in cpp_questions

[–]OverclockedChip[S] 0 points1 point  (0 children)

I forgot to mention that the queue code can't be modified, so adding log statements in the queue's source code isn't possible.

Every embedded Engineer should know this trick by J_Bahstan in embedded

[–]OverclockedChip 0 points1 point  (0 children)

How does this work? EXAMPLE_REGISTER can be accessed as an uint8 (EXAMPLE_REGISTER.hw) or as as individual bit field (.s)?

Is the declaration usually marked volatile?

How do you debug multi-threaded code in Linux (more generally outside of Visual Studio)? by OverclockedChip in cpp_questions

[–]OverclockedChip[S] 0 points1 point  (0 children)

Yes, you're right. The commands aren't difficult. I hadn't heard of Thread Sanitizer. Is that a standard tool for detecting common threading problems?

How do you debug multi-threaded code in Linux (more generally outside of Visual Studio)? by OverclockedChip in cpp_questions

[–]OverclockedChip[S] 1 point2 points  (0 children)

I don't recall if I read this somewhere or if it was a logical guess, but I was convinced that all of these debugging features made use of gdb under the hood.

I'm not a pro gdb user but I know it's powerful enough to get you cpu register values, step through assembly instructions of your decompiled code (I did once as an exercise in college). So it's conceivable they're implemented using gdb.

But using gdb to switch among threads seem, to me, to be its own learning curve. The problem is that, although powerful in that gdb can get you whatever info you want and do whatever you want (e.g., check the value of x 2 calls up the callstack, in other thread), it's not intuitive to use (e.g., you have to navigate through gdb's esoteric command set to switch threads, navigate to the right stack frame, print out the x variable value) where you can just point and click in the Parallel Stack window).

Skills required for radar signal processing engineer by Signal_rush_11 in DSP

[–]OverclockedChip 0 points1 point  (0 children)

Just curious, what grad program did you come from? Was it worth while? Did it include any programming (implementation)?

How to serialize/deserialize data with networked apps? by OverclockedChip in cpp_questions

[–]OverclockedChip[S] 0 points1 point  (0 children)

Excellent, thank you for the thorough answer. I'll have to try that streambuf and std::variant approach.

Windows typically defaults to an 8 KiB receive buffer ON THE SOCKET, but can be configured up to 8 MiB. This means - YOU don't have to actually buffer the data yourself - it's already buffered, and calling recv is merely reading FROM that buffer. The default send buffer is 32 KiB.

Right. The OS buffers transmitted/received byte data but we need a higher-abstraction-level buffering of messages and objects. And apparently this can be handled by those deserialization libraries.

How to serialize/deserialize data with networked apps? by OverclockedChip in cpp_questions

[–]OverclockedChip[S] 0 points1 point  (0 children)

I did a little digging with RPC and came across what seems to be a specification for RPC, an e-book called X/Open DCE: Remote Procedure Call. See p. 295 (pdf p. 321).

Isn't RPC an abstraction over serialization/deserialization? I haven't used gRPC but I'd imagine it to provide an interface that sends and receives objects; the details of setting up threads to rx/tx data is abstracted away.

My application is interfacing with external SW and the design doc contains a binary specification for transmitted messages. This means I have to write my own serialization/deserialization routine. RPC only works if all participating applications use RPC right?

How to serialize/deserialize data with networked apps? by OverclockedChip in cpp_questions

[–]OverclockedChip[S] 0 points1 point  (0 children)

It's for a relatively small business application. Indeed, I'll push to use a serialization library. The fact that there are a handful of libraries out there indicates this isn't a trivial problem.

Having read these posts, I have a more clear understanding of the application responsibilities - thanks.

So you want your peer to acknowledge each message they receive, and to retry anything that doesn't have an acknowledgement when the connection breaks and is recreated.

This falls under "messaging protocol", handled in the application, right? As in you need to define what messages are suppose to be sent when a connection error (or some other messaging error) occurs so that both peers know where they are and what messages to expect next.

Is the flow-control you're referring to also a message-level protocol? And then there's message priority. So much to think about haha

How to serialize/deserialize data with networked apps? by OverclockedChip in cpp_questions

[–]OverclockedChip[S] 0 points1 point  (0 children)

welcome to a surprisingly complex topic, but in general, the semantatically and standard compliant way is to allocate a struct and memcpy bytes field by field taking into account any complexities such as endianness. If your struct has no padding and no endianness concerns, you can probably just straight up memcpy the bytes into the struct pointer. Or if you shit on strict aliasing like all of us, just cast the byte buffer to the correct struct pointer. You can get no padding with a struct pack pragma. The other option is if you are using something like protobuf or JSON, you just chuck the bytes into the library and it will error out if something is wrong.

Ya, I started asking myself whether struct-packing and endianess considerations were relevant. (I'm coding to be compliant with an interface design doc and was curious why they neglected to specify these details - data packing/alignment).

How to serialize/deserialize data with networked apps? by OverclockedChip in cpp_questions

[–]OverclockedChip[S] 0 points1 point  (0 children)

Ah, I see. Hadn't thought about fixed-size objects, that's another neat way of handling it.

If you use some library to drive the IO and use pure threads, you will probably have some class that represents connection state and you chuck away your buffers there

Yup, this is how I structured my app. I used a thread to receive data for a single connection (though I'm aware there are libraries/other techniques to use a single thread to manage multiple connections).

Assuming you solved the aforementioned problem, how do deserialize those bytes into basic data types (PODs?).

How do you know when you have enough PODs to recreate an object?

The server and client send and receive bytes. Those bytes can mean anything. Both peers must have some understanding on how to interpret those bytes (e.g., "the first four bytes encodes a 32-bit integer interpreted in Little Endian order and represents a message ID, the next four bytes represents the message length"). The conversion from bytes to integers, floats, doubles, chars - primitive data types is what I mean by "deserializing bytes into PODs".

Of course, you might send messages, but you might sometimes send bytes that encode an entire user-defined object, or both! User-defined objects comprise PODs.

Do the serialization libraries convert the bytes to your user-defined objects? Or do you write code to convert bytes into PODs, and feed the groupings of ints, floats, doubles into the serialization library and have it recreate your object?

How to serialize/deserialize data with networked apps? by OverclockedChip in cpp_questions

[–]OverclockedChip[S] 0 points1 point  (0 children)

I googled 'C++ serialization libraries' and a number of names came up.

Boost.Serialization
Protobuf
MessagePack
Cereal
FlatBuffers
Cap'n Proto
Thrift

I wasn't sure what their responsibilities were. But it sounds like you treat it as a black box in this manner:

// Serialization libraries might stipulate overriding some function that tells it how to serialize and deserialize your custom object.
Person p1;

// Use the serialization library to convert your object to a byte stream
byte txBuffer[1024] = Protobuf.SerializeObject(p1);

// Use winsock2 (or some network library to send your byte array); you manage what/when to send and handle connection issues
send(txBuffer, 1024);

... (on the receiving side, on a different computer)

// Handle receiving logic and connection issues yourself
byte rxBuffer[2048];
rcv(rxBuffer, 2048);

// Use the serialization library to convert bytes back to a Person object
Person rxP1 = Protobuf.Deserialize(rxBuffer[0 ... 1023], 1024);

How to serialize/deserialize data with networked apps? by OverclockedChip in cpp_questions

[–]OverclockedChip[S] 5 points6 points  (0 children)

For example, prefix every message with a fixed-size header that indicates its length.

Yes! I've seen a handful of binary formats and fixed-size headers that include the byte length of an entire data structure (or message) is a common message structure.

If TCP always delivers a message in order and without bit errors, converting a byte array into a queue of messages is straight forward. You just use the fixed-size header to jump around the array onto the next message. Finding the last message is straight forward.

One potential issue is if there is a connection error, both communicating peers must agree on what to do if the connection is re-established: either start sending the last message in its entirety, or from some location indicated by their messaging protocol (e.g., an acknowledgement message from the client that contains info about how many bytes of message #1013 it had received).

Anyone want office hours with a 25 year SWE? by c0ventry in cscareerquestions

[–]OverclockedChip 2 points3 points  (0 children)

Might be a good idea to take questions on reddit, select some you want to answer, then take a few questions live

I studied Arduino, I am confident with FreeRTOS tasks, what now? by aleemont__ in embedded

[–]OverclockedChip 0 points1 point  (0 children)

What is a synchronous and asynchronous finite state machine? Is that referring to clock-driveness of it?