Topic: Why is pack expansion apparently restricted?


Author: Peter Bright <drpizza@gmail.com>
Date: Fri, 4 Dec 2009 13:14:42 CST
Raw View
I woke up abnormally early this morning and whilst tooling around on
the Internet, I noticed the g++ I had installed on my linux shell
supported variadic templates.

So, interesting stuff. It looks like they can express a generalized
zipN, which is surprisingly unusual for statically typed languages, so
I idly set about putting such a thing together. Now I'm sure that the
style I'm using isn't optimal in various ways, but I figure it kind of
shouldn't have to be; these things shouldn't be that hard to use,
surely.

So what I had was something like this:
#include <iostream>
#include <list>
#include <tuple>
#include <memory>

template<typename Function>
Function packed_for_each(Function f)
{
       return f;
}

template<typename Function, typename T, typename... Args>
Function packed_for_each(Function f, T& t, Args&... a)
{
       f(t);
       return packed_for_each(f, a...);
}

struct front_popper
{
       template<typename T>
       void operator()(T& t) { t.pop_front(); }
};

template<template<typename, typename> class Cont, typename First,
typename... Rest>
Cont<std::tuple<First, Rest...>, std::allocator<std::tuple<First,
Rest...>>> zipN(Cont<First, std::allocator<First>> first, Cont<Rest,
std::allocator<Rest>>... rest)
{
       typedef Cont<std::tuple<First, Rest...>,
std::allocator<std::tuple<First, Rest...>>> return_type;
       return_type rv;
       while(first.size() > 0)
       {
               rv.push_back(std::make_tuple(first.front(), rest.front()...));
               first.pop_front();
               packed_for_each(front_popper(), rest...);
       }
       return rv;
}

template <int N, int M, typename T>
struct printer_unit
{
       static std::ostream& do_it(std::ostream& os,const T& t)
       {
               os << ", " << std::get<N>(t);
               return printer_unit<N + 1, M, T>::do_it(os,t);
       }
};

template <int N, typename T>
struct printer_unit<0, N, T>
{
       static std::ostream& do_it(std::ostream& os,const T& t)
       {
               os << std::get<0>(t);
               return printer_unit<1, N, T>::do_it(os,t);
       }
};

template <int N, typename T>
struct printer_unit<N, N, T>
{
       static std::ostream& do_it(std::ostream& os,const T& t)
       {
               return os;
       }
};

template <typename...  T>
std::ostream& print_tuple(std::ostream &os, const std::tuple<T...> &t)
{
       return printer_unit<0, sizeof...(T), std::tuple<T...>>::do_it(os,t);
}

template<typename... Types>
std::ostream& operator<<(std::ostream& os, const std::tuple<Types...>&
t)
{
       return print_tuple(os, t);
}

int main()
{
       std::list<int> ints; ints.push_back(1); ints.push_back(2);
ints.push_back(3);
       std::list<char> chars; chars.push_back('a'); chars.push_back('b');
chars.push_back('c');
       std::list<float> floats; floats.push_back(3.14f); floats.push_back
(2.71f); floats.push_back(1.41f);

       std::list<std::tuple<int, char, float>> rv = zipN(ints, chars,
floats);
       for(std::list<std::tuple<int, char, float>>::const_iterator it
(rv.begin()), end(rv.end()); it != end; ++it)
       {
               std::cout << *it << std::endl;
       }
}

It works, in the version of g++ I have, but it all feels a bit, well,
suboptimal. And like I say, I'm sure a better style would help (I
dunno what the preferred/envisaged idioms are for this kind of thing),
but equally I feel that there are some little things that could be
done to make it easier to write.

The big one is the pack expansion patterns. The overall system is kind
of neat and the pattern-based generation feels very natural, insofar
as it appears to do the right thing in this line:
rv.push_back(std::make_tuple(first.front(), rest.front()...));

What's less satisfactory is calling pop_front. Pack expansion
apparently doesn't occur in a bare statement. I can't write:
rest.pop_front()...;
because that is not one of the blessed sites for expansion. In fact, I
can't see how function calls are even permitted (they don't seem to be
one of initializer-list, base-specifier-list, mem-initializer-list,
template-argument-list, exception-specification, or attribute-list),
but common sense suggests that they must be.

Indeed, the entire reason that I had to have a First/Rest...
separation is because I needed to have a non-packed reference to at
least one thing, because I can't say:
while(rest.size() > 0...)
or anything like that. It's a pity, because I would like to. Sure, I
imagine for this that I could write something to produce the same
effect for me, but it doesn't seem unreasonable for pack expansion to
work in this situation too, using && to join conditions (I suppose the
argument would be that sometimes you want &&, sometimes || might
suffice, but really it would be more useful to pick one and have it
useful at least some of the time than to have neither, and since you'd
have no idea which one failed, && seems the only sensible option).

Of course, copying and subsequently destroying the lists just to zip
them up isn't really ideal, but absent some hypothetical construct
letting me write:
template<template<typename, typename> class Cont, typename... Types>
Cont<std::tuple<Types...>, std::allocator<std::tuple<Types...>>> zipN
(Cont<Types, std::allocator<Types>>... values)
{
       typedef Cont<Types, std::allocator<Types>>... arg_pack;
       Cont<std::tuple<Types...>, std::allocator<std::tuple<Types...>>>
return_type;
       return_type rv;
       for(typename arg_pack::iterator... it(values.begin())..., end
(values.end())...; it... != end...; ++it...)
       {
               rv.push_back(make_tuple(*it...));
       }
       return_type rv;
}
it seems the simplest way of solving the problem. I'm envisaging ...
pattern expansion as making the code kind of "multidimensional", so
it'll iterate through the arbitrarily many lists in lock-step.

I'm sure something clever could be done with recursive templates, but
I'm not sure I should have to write something clever with recursive
templates....

Is g++ just being too conservative about what it lets me do, or is it
actually prohibited by the spec, and if the latter, is there any good
reason for it that I am missing?

--
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@netlab.cs.rpi.edu]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: Larry <cppljevans@gmail.com>
Date: Sat, 5 Dec 2009 01:02:09 CST
Raw View
On Dec 4, 1:14 pm, Peter Bright <drpi...@gmail.com> wrote:
> I woke up abnormally early this morning and whilst tooling around on
> the Internet, I noticed the g++ I had installed on my linux shell
> supported variadic templates.
>
> So, interesting stuff. It looks like they can express a generalized
> zipN, which is surprisingly unusual for statically typed languages, so
> I idly set about putting such a thing together. Now I'm sure that the
> style I'm using isn't optimal in various ways, but I figure it kind of
> shouldn't have to be; these things shouldn't be that hard to use,
> surely.
>
> So what I had was something like this:
> #include <iostream>
> #include <list>
> #include <tuple>
> #include <memory>
>
> template<typename Function>
> Function packed_for_each(Function f)
> {
>        return f;
>
> }
>
> template<typename Function, typename T, typename... Args>
> Function packed_for_each(Function f, T& t, Args&... a)
> {
>        f(t);
>        return packed_for_each(f, a...);
>
> }
>

I believe this can be done with:

template<typename Function, typename... Args>
  Function packed_for_each(Function f, Args&... a)
  {
         f(a)...;
  }

At least if I understand p. 14 of:

   http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2242.pdf

In particular, the code:

template<typename... Args> void g(Args... args)
{
    f(const cast<const Args*>(&args)...); // okay: ''Args'' and
''args'' are expanded
    f(5 ...); // error: pattern does not contain any parameter packs
    f(args); // error: parameter pack "args" is not expanded
    f(h(args...) + args...); // okay: first ''args'' expanded within h,
second ''args'' expanded within f.
}

HTH.

-Larry

--
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@netlab.cs.rpi.edu]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: Peter Bright <drpizza@gmail.com>
Date: Sat, 5 Dec 2009 22:31:35 CST
Raw View
On Dec 5, 7:02 am, Larry <cppljev...@gmail.com> wrote:
> On Dec 4, 1:14 pm, Peter Bright <drpi...@gmail.com> wrote:
>
>
>
>
>
> > I woke up abnormally early this morning and whilst tooling around on
> > the Internet, I noticed the g++ I had installed on my linux shell
> > supported variadic templates.
>
> > So, interesting stuff. It looks like they can express a generalized
> > zipN, which is surprisingly unusual for statically typed languages, so
> > I idly set about putting such a thing together. Now I'm sure that the
> > style I'm using isn't optimal in various ways, but I figure it kind of
> > shouldn't have to be; these things shouldn't be that hard to use,
> > surely.
>
> > So what I had was something like this:
> > #include <iostream>
> > #include <list>
> > #include <tuple>
> > #include <memory>
>
> > template<typename Function>
> > Function packed_for_each(Function f)
> > {
> >        return f;
>
> > }
>
> > template<typename Function, typename T, typename... Args>
> > Function packed_for_each(Function f, T& t, Args&... a)
> > {
> >        f(t);
> >        return packed_for_each(f, a...);
>
> > }
>
> I believe this can be done with:
>
> template<typename Function, typename... Args>
>   Function packed_for_each(Function f, Args&... a)
>   {
>          f(a)...;
>   }
>
> At least if I understand p. 14 of:
>
>    http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2242.pdf
>
> In particular, the code:
>
> template<typename... Args> void g(Args... args)
> {
>     f(const cast<const Args*>(&args)...); // okay: ''Args'' and
> ''args'' are expanded
>     f(5 ...); // error: pattern does not contain any parameter packs
>     f(args); // error: parameter pack "args" is not expanded
>     f(h(args...) + args...); // okay: first ''args'' expanded within h,
> second ''args'' expanded within f.
>
> }
>

The version of g++ I'm on doesn't like that; I notice that in all the
examples, the ... are inside the function call brackets, not outside.
It seems that expansion only occurs in a function call context  (or a
template argument context, a base class context, etc.)--i.e. it can
fabricate function arguments joined with commas, but it can't
fabricate expressions joined with commas or statements joined with
semicolons.

For functions returning values, I can do something like this:

template<typename... Args>
void ignore(const Args&...)
{
}

// ...

ignore(pack.some_function_with_a_return_value()...);

which isn't ideal, but works. However, for void returning functions
that's apparently not an option. Which kinda makes sense, but is
annoying nonetheless, since it means void returns must be special-
cased. And I imagine allowing declarations of the type void foo(void,
void, void) (etc.) is a bridge too far. It's a pity void is not a true
unit type with a single inhabitant, which would avoid this problem, as
it would make void arguments (and void members of structures, etc.)
legal, and put it on an equal standing with other types.



--
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@netlab.cs.rpi.edu]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: Larry Evans <cppljevans@gmail.com>
Date: Sun, 6 Dec 2009 12:15:54 CST
Raw View
On Dec 5, 10:31 pm, Peter Bright <drpi...@gmail.com> wrote:
> On Dec 5, 7:02 am, Larry <cppljev...@gmail.com> wrote:
[snip]
> > I believe this can be done with:
>
> > template<typename Function, typename... Args>
> >   Function packed_for_each(Function f, Args&... a)
> >   {
> >          f(a)...;
> >   }
>
[snip]
> The version of g++ I'm on doesn't like that; I notice that in all the
> examples, the ... are inside the function call brackets, not outside.

Ah!  I actually tried:

 f(a)...

and got the error which I guess you're getting:

 packed_for_each.cpp: In function 'Function packed_for_each(Function,
Args &
         ...
     )':
 packed_for_each.cpp:32: error: expected ';' before '...' token
 packed_for_each.cpp:36: error: parameter packs not expanded with
'...':
 packed_for_each.cpp:36:         'a'

> It seems that expansion only occurs in a function call context  (or a
> template argument context, a base class context, etc.)--i.e. it can
> fabricate function arguments joined with commas, but it can't
> fabricate expressions joined with commas or statements joined with
> semicolons.
>
> For functions returning values, I can do something like this:
>
> template<typename... Args>
> void ignore(const Args&...)
> {
>
> }
>
> // ...
>
> ignore(pack.some_function_with_a_return_value()...);
>

Taking your suggestion, I tried this:

template<typename... Args>
void ignore(Args...)
{
}

template<typename Function, typename... Args>
Function packed_for_each(Function f, Args&... a)
{
      ignore(f(a)...);
      return f;
}

struct void_
{
};

struct front_popper
{
      template<typename T>
      void_ operator()(T& t)
      {
          t.pop_front();
          return void_();
      }
};

and it now compiles.

[snip]

> It's a pity void is not a true
> unit type with a single inhabitant, which would avoid this problem, as
> it would make void arguments (and void members of structures, etc.)
> legal, and put it on an equal standing with other types.

Agreed.  The unit could be something like the void_ in the above
snippet.

-regards,
Larry


--
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@netlab.cs.rpi.edu]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: Larry Evans <cppljevans@gmail.com>
Date: Sun, 6 Dec 2009 12:17:54 CST
Raw View
On Dec 5, 10:31 pm, Peter Bright <drpi...@gmail.com> wrote:
[snip]
> The version of g++ I'm on doesn't like that; I notice that in all the
> examples, the ... are inside the function call brackets, not outside.
> It seems that expansion only occurs in a function call context  (or a
> template argument context, a base class context, etc.)--i.e. it can
> fabricate function arguments joined with commas, but it can't
> fabricate expressions joined with commas or statements joined with
> semicolons.

Have you tried a later version of the compiler?  I'm using:

ftp://gcc.gnu.org/pub/gcc/snapshots/4.4-20090630/gcc-4.4-20090630.tar.bz2

and it compiles:

 template
 < typename Layout
 , index_type... Indices
>
 struct
destroyers
 < Layout
 , mpl::package_c<index_type,Indices...>
>
{
       static
     destroy_fun
   _( index_type index_valu)
   {
           static
         destroy_fun const
       vec
      [ sizeof...(Indices)
         ]
       = { Layout::destroyer(index_wrap<Indices>())... //<==expansion
         };
       return vec[index_valu-index_undefined];
   }

};

from:

http://svn.boost.org/svn/boost/sandbox/variadic_templates/boost/composite_tagged.hpp

HTH.

-Larry


--
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@netlab.cs.rpi.edu]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: Peter Bright <drpizza@gmail.com>
Date: Tue, 8 Dec 2009 01:32:11 CST
Raw View
On Dec 6, 6:15 pm, Larry Evans <cppljev...@gmail.com> wrote:
> > It's a pity void is not a true
> > unit type with a single inhabitant, which would avoid this problem, as
> > it would make void arguments (and void members of structures, etc.)
> > legal, and put it on an equal standing with other types.
>
> Agreed.  The unit could be something like the void_ in the above
> snippet.

I don't think it's possible to get the right semantics using only
language features. A unit type needs to be semantically a singleton
(i.e. it has only a single inhabitant), but in order to be useful it
needs to be syntactically a regular type. i.e. given:
unit u1 = unit();
unit u2 = unit();

&u1 needs to be identical to &u2, which isn't something that C++ can
achieve on its own. It needs to be built-in.

Though templates did introduce some new features for void (such as
void returns), it is still some peculiar thing with inconsistent
properties, with the result that we need to handle it inconsistently.

Of course it's not something that can ever be fixed, due to the
cancerous C compatibility desire.


--
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@netlab.cs.rpi.edu]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]





Author: Peter Bright <drpizza@gmail.com>
Date: Tue, 8 Dec 2009 01:31:42 CST
Raw View
On Dec 6, 6:17 pm, Larry Evans <cppljev...@gmail.com> wrote:
>  template
>  < typename Layout
>  , index_type... Indices
>
>  struct
> destroyers
>  < Layout
>  , mpl::package_c<index_type,Indices...>
>
> {
>        static
>      destroy_fun
>    _( index_type index_valu)
>    {
>            static
>          destroy_fun const
>        vec
>       [ sizeof...(Indices)
>          ]
>        = { Layout::destroyer(index_wrap<Indices>())... //<==expansion
>          };
>        return vec[index_valu-index_undefined];
>    }
>
> };

That's in an initializer-list context; it's still not a bare function
call. Initializer lists are one of the blessed expansion sites.


--
[ comp.std.c++ is moderated.  To submit articles, try just posting with ]
[ your news-reader.  If that fails, use mailto:std-c++@netlab.cs.rpi.edu]
[              --- Please see the FAQ before posting. ---               ]
[ FAQ: http://www.comeaucomputing.com/csc/faq.html                      ]