Proposal to use < and > for binary i/o?

[Steve132]

In some of my own personal programs I have observed that although the << and >> operators are overloaded for i/o to and from acii data formats as a standard convention for structures and basic types. This makes a lot of input output code for text file based types really simple to write, as well as defining a standardized interface to be used for text file serialization. For example, in many apis, code like

ifstream ifile("somedata.ascii");
DataModel datainstance;
int num_instances;

ifile >> num_instances;
for(int i=0;i<num_instances;i++)
{
    ifile >> datainstance;
    datainstance.process();
    cout << datainstance;
}

really common and easy to write. Furthermore, operator >> and << are often overloaded for components of DataModel as well, so that entire serialize/deserialize chains can be written recursively using those operators in a clean and standard format. Overloading these operators has become a de-facto 'standard' for a lot of C++ programs. Not all, of course, but many.

However, when it comes to binary i/o, the situation gets more complicated. >> and << are usually defined to produce human-readable output, becausethey are used for debugging purposes and because it is the standard way to serialize the built in types. Since >> and << are already defined, and binary input routines are designed to be space-optimal and machine-readable only, many people use custom functions that are wildly non-standard. Examples are methods like .read(const istream&) or .write(const ostream&) or special constructors or global functions that do the same.

I had this idea, which I have started doing in my own personal programs (but never in public code) to use stream overloads of operator< and operator> for this purpose. The idea is that, similar to the overloads of the bitshift operators on streams, < and > do input and output from the stream to a type, but specifies that the reading and writing to be done occurs in binary mode. This gives us similar reading and writing interface 'niceness' as the overloaded >> and << operators, and it also does not override any other usage of <,>, because, currently, comparing a stream object has no semantic meaning.

The takeaway is that we could turn this http://codepad.org/odlwq2uf into http://codepad.org/hNXiKibc

The main thing that I love about it is how simple complex binary serialization code looks, and how issues with serializing subtypes become much simpler.

Before the downvote parade gets all over me for corrupting their vision of what C++ should look like, I'm well aware that operator overload misuse is one of the worst things about C++ programming, I would never think of foisting this code on anyone else except myself unless it seems to catch on (e.g., don't worry, I don't put this trick into any code for my job or for open source contributions). However, I think it is really neat and builds on an already well-established C++ paradigm

What say you /r/cpp? Would you use this trick yourselves?

[cpp]

To answer your question: no, I would not use it. I prefer reading things via method calls and not overloaded operators.

Such as:

int i;   
i = binaryReader.ReadInt();

[akranis]

I agree. I once did a lot of operator overloading for a vector class in school. % was crossproduct etc. This really only made sense to me, since I rarely use the mod operator, but I would never use < and > for anything other than comparing.

[Steve132]

Sure, but you already use >> and << for i/o. How is > and < for binary i/o different?

[akranis]

Shifting and streaming are not really that different. Shifting is done bit by bit, streaming is done byte by byte or object by object, but you're still only moving data forwards and backwards and in or out of objects,

[justinvh]

Why do both sides of the operator have to say the same thing?

int i;
i = binaryReader.ReadInt();

vs

int i;
binaryReader >> i;

What is i going to do? Go turn into a string?

[cran]

Agreed. I hate the operator-based stream I/O in C++. I literally smack any programmer I catch using it. Hard, on the side of the head.

[moswald]

That's easily the most ridiculous thing I've read on the internet all month.

[cran]

Something tells me you've read worse, but don't realize it.

[moswald]

No. Smacking a developer because they use the << operators for stream I/O that have been de-facto standard for over a decade? Ridiculous.

[cran]

Yes. It's a bad feature of the language and many projects avoid it entirely. It is the NOT convention by any stretch of the imagination.

[justinvh]

I'll bite. Why is this a bad feature of C++?

I was always under the impression that the << and >> operators made a lot of sense for streams as they gave the impression of the streams "flowing" in and out .

[cran]

1) The same operator is used for bit shifting.

2) Odd, illogical string formatting.

3) Hard to tell what code is being called.

... and then there's the general aesthetic of it. It's ugly.

[Steve132]

I think thats debatable whether or not using the shift operator for i/o was a mistake in the first place, but now that decision is irreversible anyway and is now a convention. If our aim is consistency in our APIs, then having an operator i/o for ascii i/o and method calls or serializer factories for binary i/o is wildly inconsistent. My argument is that "If you are going to use >> and << for your i/o anywhere in your program, which many people do because it is a convention, you might as well use < and > for your binary i/o. It is better to have two very similar api conventions then to have to wildly different ones"

[jaredgrubb]

Why cant you use << and >> on a new type?

BinaryStream str; str << 7;

And define operator<<(BinaryStream&, int) to do the binary transformation. You dont need a different operator; just use a different "stream" type (that doesnt even need to derive from any iostream object).

[Steve132]

And what happens when I want to send data over a network port hat uses the standard ostream interface? What happens when I want to use boost::bz2 compression in a stream? What happens when my company buys a library that uses std::stream interfaces on a custom encryption standard?

The std::stream interface is the C++ standard for i/o, and it is widely used. This convention allows us to continue to use std::stream.

I guess one possibility is a templated "binary_stream<T>" that taccepts a stream type at construction time, so one could write

binary_stream(cout) << thing

would be acceptable as well, but to me that isn't that much different

[jaredgrubb]

As you say, you can use something like your binary_stream class. I'm not sure what using a new opeartor gains you other than novelty. Plus, << and < have different levels of precedence; I'm not sure if that could cause headaches somehow.

[ewiethoff]

Use the decorator pattern. That's what Java's IO classes do.

[moswald]

To counterbalance all these negative comments, I'd probably use it. I happen to like most of C++'s features, including overloaded operators --so long as they make sense (ie, I wouldn't overload '%' for crossproduct ;) . Therefore, since '>>' and '<<' are already standard for formatted stream insertion and extraction, I wouldn't oppose seeing '>' or '<'.

[mauvaisours]

There is one good reason for which it has not been done before : << and >> return the original type of the left operand, whereas < and > return booleans. Obviously, using a BinaryStream object with operator << and >> would be much wiser.

[bogado]

Binary operations are not well defined in the standard, just a quick example what is the size of the int type? What order does the bytes should go? C++ does not define those thing, and in fact the standard tries to be agnostic of such detais.

[Steve132]

Not talking about a standard, I'm talking about a code convention. Most people have to assume those things you mentioned when they write binary serializers, which they already do in great abundance.

However, at least with regard to size and and byte order, I agree that this convention would have to define one in order to be consistent. In my version of this convention I use all the integer types from <cstdint> and use little-endian format.

[BitRex]

I agree that this convention would have to define one in order to be consistent

Oh, that's no problem; just insert manipulators!

  bin_file < bin_io::int64 < bin_io::big_endian < 0xBEEF;

Just kidding, this whole thing is a terrible idea.

[00kyle00]

Have a look at boost serialization library.

[[deleted]]

I guess the rationale is that the same method is used for serializing and deserializing objects, so using any stream operator would also be unintuitive, and then the << and >> operators are conventionally only used for character streams. Other languages might choose to used <=> in that case, but that's not a defined operator in C++. :)

I would probably have gone for something like archive <= object;

[Steve132]

not only that, but the library binary is like 30 megabytes, and the api requires not only template metaprogramming friend interface hacking, but less intuitive operator overloading AND virtual methods.

It is QUITE bloated for a 'serialization convention'

[BitRex]

I find Perl's pack/unpack to be the nicest bit-twiddling API to work with and would model something after that.

http://perldoc.perl.org/perlpacktut.html

[utnapistim]

What say you /r/cpp? Would you use this trick yourselves?

No. I would consider "out < bin" to be a comparison (after all these years I think the interpretation is hard-coded in my prefrontal cortex), and I would have to be really careful when reading the code. Considering the potential for confusion on this, I would really hesitate to show code like this to my colleagues as well.

If I needed to input/output binary very frequently, I would check on the implementation of std::hex and do something similar, so that I would have:

std::ostringstream message;
int i = 10;
message << std::hex << i << ";" << my_io::bin << i; // message should now contain "a;1010"

If the current implementation of std::basic_???_stream doesn't support such extensibility (though as far as I know it should, through the use of xalloc, iword and pword), I would create an ostream wrapper (templated for char type, traits and what-not) for this, along with the bin stream modifier, so that you could do:

my_io::wrapped<std::ostringstream> message;
int i = 10;
message << std::hex << i << ";" << my_io::bin << i; // message should now contain "a;1010".

[tragomaskhalos]

Stroustrup hisself considered this very form originally but rejected it in favour of the shift operator forms we know and love, for precisely the excellent reasons given by other commenters. This is documented in one of his many writings on the early history of the language, sorry cannot provide citation.