<sirwillard@my-deja.com> wrote:

[SNIP]
> The buffer classes aren't tied to character-oriented baggage.  It's the
> streams classes that are.  For instance, the streambuf classes never
> reference std::locale.  A binary stream class could (and probably
> should) use a standard streambuf.

Yes, they are.  Look at std::basic_streambuf<>'s pubimbue, getloc, and
imbue methods.  They use locales, which wouldn't be needed for pumping
raw data around.  I don't think a binary_streambuf class would need a
char_traits either.

[SNIP]
> Multibyte for binary I/O?  Templated to any (POD?) type?

> Doesn't sound like you're talking about binary I/O any more.

Why not, it's still raw data that can be bitwise-copied.  I'm
speculating on the possiblity; only (unsigned) char would be guarrunteed
for binary I/O for certain.

--
Daryle Walker
Mac, Internet, and Video Game Junkie
dwalker07 AT snet DOT net

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

Valentin Bonnard wrote:

>
> If you are interrested by this idea, then have a look at
> my _proposed_ (in clear, not implemented) model for type
> destructuring (hum... I don't know if this word exists):
>
> <URL:http://www.eleves.ens.fr:8080/home/bonnard/Reflex++/ReflexiveC%2B%2B.html>

Hi Valentin,

    did you ever hear of this?

http://www.hlla.is.tsukuba.ac.jp/~chiba/openc++.html

those guys implemented a compiler to do C++- compile time reflection, which can be
easily used to either generate runtime reflection or can even implement object
serialization.

Ole

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

In article <3A1BF25C.5C2C8A8F@physik.tu-muenchen.de>,
  Christopher Eltschka <celtschk@physik.tu-muenchen.de> wrote:
> James.Kanze@dresdner-bank.com wrote:

> [...]

> > That said, my impression is also that using a standard streambuf
> > (std::filebuf) with an identity conversion is the correct solution
> > for the streambuf.  For the stream itself, the problem is that
> > pure binary streaming is next to worthless, and that there is no
> > real standard binary format.  What should a binary stream output:
> > IIOP, BER, RPC,... ?

> All of them.
> Just as an ASCII stream can output numbers in decimal, hexadecimal
> and octal representation, with thousand-separators or without, using
> '.' or ',' as separator, etc.

I don't think the comparison is valid.  I certainly don't want a
stream in which the first three ints are BER, and the next two RPC.
ASCII streams allows a variety of formatting possibilities, because a
variety of formatting possibilities are allowed by the stream format.
(I could conceive of cases where this is also true for binary
streams.  At least one stream I've seen -- I forget which one -- uses
two different formats for integers.)

A more exact analogy would be a stream in ASCII, or in EBCDIC,
or maybe a stream which formatted using arabic characters (and arabic
digits) rather than the roman characters and European digit
characters.

> That is, a binary stream would also have a "locale", which in this
> case would probably be named "output_format" or similar. This would
> do all the formatting.

A locale would seem more reasonable.  But it is less than obvious to
make work, since every overloaded operator<< must be involved.

> BTW, on systems with memory mapped files, you can get a "poor man's
> persistency" by just using placement new in such a memory mapped
> file.  Of course, if you use it for anything but PODs, you might get
> problems.

Even with POD's, you might get problems.  Different versions of the
compiler might pad differently (really experienced).  About all true
binary dump files are good for is temporaries, that will be discarded
when the processes terminates.

--
James Kanze                               mailto:kanze@gabi-soft.de
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
Ziegelh   ttenweg 17a, 60598 Frankfurt, Germany Tel. +49(069)63198627


Sent via Deja.com http://www.deja.com/
Before you buy.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

sirwillard@my-deja.com wrote:
>
> In article <1ekao4f.1d7tjt1w48paaN%dwalker07@snet.net.invalid>,
>   dwalker07@snet.net.invalid (Daryle Walker) wrote:
> > A post in the Boost mailing list sparked my memory of something that
> > bothered me about the C++ stream library: it's all character based;
> > there's no binary streams.  Maybe this design hole should be closed up
> > with a binary stream buffer class.  It would be like std::streambuf,
> but
> > none of the character-oriented baggage like locales would be included.
>
> The buffer classes aren't tied to character-oriented baggage.  It's the
> streams classes that are.  For instance, the streambuf classes never
> reference std::locale.  A binary stream class could (and probably
> should) use a standard streambuf.
>
> > Users generally use the binary streambuf class directly to read/write
> > bytes from/to a stream.  Since users shouldn't need the
> > character-oriented stuff, the current I/O headers shouldn't be
> #included
> > (at least in the base class).  However, some std::streambuf classes
> > could be built on top of a binary streambuf, like filestreambuf.
>
> If you're going to use the streambuf class directly you can do that
> today.  Heck, you can even use the standard iostreams to do this.
>
> > Binary versions of the stream classes (those with << and/or >>) could
> be
> > made.  I don't know if we should mandate some sort of multi-byte
> > handling for them.  I also don't know if this binary streambuf should
> be
> > limited to (unsigned) char, or should it be templated to any (POD?)
> > type.
>
> Multibyte for binary I/O?  Templated to any (POD?) type?
>
> Doesn't sound like you're talking about binary I/O any more.

Oh yes, it does. What he thinks of is probably the equivalent
of Pascal's "file of foo" type. That is, the file is treated as
a sequence of foo.
The problem with doing this in library is that it isn't possible
to do it in an efficient way - you'd basically get the same result
as doing a s.write(&foo, sizeof(foo))) - that is, the content is
dumped to the file, as is, including all padding, without any
support for binary formatting (no, that's not a contradiction;
binary formatting consists of things like specifying the byte
order, character encodings for embedded strings - UFT8, UTF-16,
UCS-4 -, defining whether strings are to be zero terminated,
lenth prefixed, or if the length is determined some other way, ...).

To do efficient binary io libraries, you'd need an additional
language mechanism: implicit "decomposition" of PODs.
A possible syntax could be:

decompose(podvar) foo(podvar);

If podvar is a builtin type, an enumeration or an union, this would
be equivalent to foo(podvar); if it's a struct, then it would be
equivalent to a series of

decompose(podvar.member1) foo(podvar.member1);
...
decompose(podvar.membern) foo(podvar.membern);

Indeed, it would be even more useful, if it is not restricted to PODs,
with the following adapted and more formalized rules:

A decompose statement is written in the form

decompose(variable) expression;

where the expression shall depend on the variable.
A decompose statement is interpreted as follows:

- If the expression is legal as is, the decompose statement is
equivalent to
  the block statement { expression; }.
- Otherwise, if the variable is of an aggregate type, then the decompose
  statement is equivalent to the block statement
  {
    decompose(variable.member1) replacement-expression1;
    decompose(variable.member2) replacement-expression2;
    ...
    decompose(variable.membern) replacement-expressionn;
  }
  where replacement-expressionn is obtained by replacing each occurance
of the
  variable in the original expression with variable.membern. In this
case,
  the original decompose statement is legal iff each of the replacement
  statements is legal.
- Otherwise the statement is illegal.

Examples:

struct foo { int i; void* j; };
struct bar { int i; void* j; };
struct baz {};
strict xyz { xyz(); int i; int j; int k; }

void f(int);
void f(foo);
void f(void*);
void f(std::string);

void example()
{
  int i = 0;
  decompose(i) f(i);
  // since f(i) is a legal expression (calling void f(int)),
  // this is equivalent to { f(i); }

  char* s = "Hello";
  decompose(s) f(s);
  // since f(s) is a legal expression (calling void f(void*)),
  // this is equivalent to { f(s); }

  foo F;
  decompose(F) f(F);
  // there's an overload taking foo, therefore this is { f(F); }

  bar b;
  decompose(b) f(b);
  // f(b) would be illegal (no matching function), but b is an
aggregate,
  // therefore this is equivalent to
  // { decompose(b.i) f(b.i); decompose(b.j) f(b.j); }
  // Each of those replacement decomposes is legal according point 1,
and
  // therefore this expands to { { f(b.i); f(b.j); }

  baz B;
  decompose(B) f(B);
  // again, f(B) doesn't work; decomposition results in the empty block
  // statement {}, which certainly is legal

  xyz x;
  decompose(x) f(x);
  // f(x) is an error, and x is not an aggregate (constructor!),
therefore
  // this line is ill-formed.

  std::string s;
  decompose(s) f(s);
  // since f(s) is legal, this is equivalent to { f(s); }
}

Such a feature could be used f.ex. in writing to/reading from binary
streams:

void write(binstream& bs, int i);
void write(binstream& bs, double d);
void write(binstream& bs, complex<double> c);

void read(binstream& bs, int& i);
void read(binstream& bs, double& d);
void read(binstream& bs, complex<double>& c);

class Foo
{
public:
  Foo(binstream& bs)
  {
    decompose(members) read(bs, members);
  }
  void store(binstream& bs) const
  {
    decompose(members) write(bs, members);
  }
private:
  struct
  {
    int foo, bar, baz;
    double foobar;
  } members;

But it would also help f.ex. in debugging output:

  // still inside Foo:

public:
  void debug_print(std::ostream& os)
  {
    os << "Foo (" << (void*)this << ") members: { ";
    decompose(members) os << members << " ";
    os << "}";
  }
};

And there are other possible uses I can think of as well.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

> To do efficient binary io libraries, you'd need an additional
> language mechanism: implicit "decomposition" of PODs.

If you are interrested by this idea, then have a look at
my _proposed_ (in clear, not implemented) model for type
destructuring (hum... I don't know if this word exists):

<URL:http://www.eleves.ens.fr:8080/home/bonnard/Reflex++/ReflexiveC%2B%2B.html>

I use it implement IO:

<URL:http://www.eleves.ens.fr:8080/home/bonnard/Reflex++/Serial++.html>

--

Valentin Bonnard

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

sirwillard@my-deja.com wrote:
> In article <1ekao4f.1d7tjt1w48paaN%dwalker07@snet.net.invalid>,
>   dwalker07@snet.net.invalid (Daryle Walker) wrote:
>> A post in the Boost mailing list sparked my memory of something that
>> bothered me about the C++ stream library: it's all character based;
>> there's no binary streams.

Isn't streambuf usable for this purpose   ?

>> Maybe this design hole should be closed up
>> with a binary stream buffer class.  It would be like std::streambuf,
> but
>> none of the character-oriented baggage like locales would be included.
>
> The buffer classes aren't tied to character-oriented baggage.  It's the
> streams classes that are.  For instance, the streambuf classes never
> reference std::locale.

If is was so, how would they convert, for ex. Unicode to UTF-8   ?

--

Valentin Bonnard

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

In article <8vc1ve$ukt$1@nnrp1.deja.com>,
  sirwillard@my-deja.com wrote:
> In article <1ekao4f.1d7tjt1w48paaN%dwalker07@snet.net.invalid>,
>   dwalker07@snet.net.invalid (Daryle Walker) wrote:
> > A post in the Boost mailing list sparked my memory of something
> > that bothered me about the C++ stream library: it's all character
> > based; there's no binary streams.  Maybe this design hole should
> > be closed up with a binary stream buffer class.  It would be like
> > std::streambuf, but none of the character-oriented baggage like
> > locales would be included.

> The buffer classes aren't tied to character-oriented baggage.  It's
> the streams classes that are.  For instance, the streambuf classes
> never reference std::locale.

Are you sure?  The base class has an imbue function, to set the
locale, and the filebuf derived class is defined as using the codecvt
facet to convert byte oriented input and output to the actual
character type (see 27.8.1.4).

> A binary stream class could (and probably should) use a standard
> streambuf.

> > Users generally use the binary streambuf class directly to
> > read/write bytes from/to a stream.  Since users shouldn't need the
> > character-oriented stuff, the current I/O headers shouldn't be
> > #included (at least in the base class).  However, some
> > std::streambuf classes could be built on top of a binary
> > streambuf, like filestreambuf.

> If you're going to use the streambuf class directly you can do that
> today.  Heck, you can even use the standard iostreams to do this.

> > Binary versions of the stream classes (those with << and/or >>)
> > could be made.  I don't know if we should mandate some sort of
> > multi-byte handling for them.  I also don't know if this binary
> > streambuf should be limited to (unsigned) char, or should it be
> > templated to any (POD?)  type.

> Multibyte for binary I/O?  Templated to any (POD?) type?

> Doesn't sound like you're talking about binary I/O any more.

He's asking a question.

The current situation is that all IO through basic_filebuf is byte
oriented, regardless of the type basic_filebuf is instantiated over.
And the codecvt facet is used for conversion.

That said, my impression is also that using a standard streambuf
(std::filebuf) with an identity conversion is the correct solution for
the streambuf.  For the stream itself, the problem is that pure binary
streaming is next to worthless, and that there is no real standard
binary format.  What should a binary stream output: IIOP, BER,
RPC,... ?

--
James Kanze                               mailto:kanze@gabi-soft.de
Conseils en informatique orient   e objet/
                   Beratung in objektorientierter Datenverarbeitung
Ziegelh   ttenweg 17a, 60598 Frankfurt, Germany Tel. +49(069)63198627


Sent via Deja.com http://www.deja.com/
Before you buy.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

James.Kanze@dresdner-bank.com wrote:

[...]

> That said, my impression is also that using a standard streambuf
> (std::filebuf) with an identity conversion is the correct solution for
> the streambuf.  For the stream itself, the problem is that pure binary
> streaming is next to worthless, and that there is no real standard
> binary format.  What should a binary stream output: IIOP, BER,
> RPC,... ?

All of them.
Just as an ASCII stream can output numbers in decimal, hexadecimal and
octal representation, with thousand-separators or without, using '.' or
',' as separator, etc.
That is, a binary stream would also have a "locale", which in this case
would probably be named "output_format" or similar. This would do all
the formatting.

BTW, on systems with memory mapped files, you can get a "poor man's
persistency" by just using placement new in such a memory mapped file.
Of course, if you use it for anything but PODs, you might get
problems.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

A post in the Boost mailing list sparked my memory of something that
bothered me about the C++ stream library: it's all character based;
there's no binary streams.  Maybe this design hole should be closed up
with a binary stream buffer class.  It would be like std::streambuf, but
none of the character-oriented baggage like locales would be included.

Users generally use the binary streambuf class directly to read/write
bytes from/to a stream.  Since users shouldn't need the
character-oriented stuff, the current I/O headers shouldn't be #included
(at least in the base class).  However, some std::streambuf classes
could be built on top of a binary streambuf, like filestreambuf.

Binary versions of the stream classes (those with << and/or >>) could be
made.  I don't know if we should mandate some sort of multi-byte
handling for them.  I also don't know if this binary streambuf should be
limited to (unsigned) char, or should it be templated to any (POD?)
type.

--
Daryle Walker
Mac, Internet, and Video Game Junkie
dwalker07 AT snet DOT net

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]

In article <1ekao4f.1d7tjt1w48paaN%dwalker07@snet.net.invalid>,
  dwalker07@snet.net.invalid (Daryle Walker) wrote:
> A post in the Boost mailing list sparked my memory of something that
> bothered me about the C++ stream library: it's all character based;
> there's no binary streams.  Maybe this design hole should be closed up
> with a binary stream buffer class.  It would be like std::streambuf,
but
> none of the character-oriented baggage like locales would be included.

The buffer classes aren't tied to character-oriented baggage.  It's the
streams classes that are.  For instance, the streambuf classes never
reference std::locale.  A binary stream class could (and probably
should) use a standard streambuf.

> Users generally use the binary streambuf class directly to read/write
> bytes from/to a stream.  Since users shouldn't need the
> character-oriented stuff, the current I/O headers shouldn't be
#included
> (at least in the base class).  However, some std::streambuf classes
> could be built on top of a binary streambuf, like filestreambuf.

If you're going to use the streambuf class directly you can do that
today.  Heck, you can even use the standard iostreams to do this.

> Binary versions of the stream classes (those with << and/or >>) could
be
> made.  I don't know if we should mandate some sort of multi-byte
> handling for them.  I also don't know if this binary streambuf should
be
> limited to (unsigned) char, or should it be templated to any (POD?)
> type.

Multibyte for binary I/O?  Templated to any (POD?) type?

Doesn't sound like you're talking about binary I/O any more.

--
William E. Kempf
Software Engineer, MS Windows Programmer


Sent via Deja.com http://www.deja.com/
Before you buy.

---
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@ncar.ucar.edu    ]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.research.att.com/~austern/csc/faq.html                ]
[ Note that the FAQ URL has changed!  Please update your bookmarks.     ]