Topic: Modules are hard for build tools
Author: Abyx <fl.lllj@gmail.com>
Date: Fri, 26 Feb 2016 07:23:56 -0800 (PST)
Raw View
------=_Part_99_1784697087.1456500236418
Content-Type: multipart/alternative;
boundary="----=_Part_100_446552970.1456500236420"
------=_Part_100_446552970.1456500236420
Content-Type: text/plain; charset=UTF-8
Without modules we have the following situation:
There is a bunch of source files with code -
tu1.cpp, tu2.cpp, ..., tuN.cpp,
h1.hpp, h2.hpp, ..., hM.hpp,
And a build tool script (*make, gyp, gn, etc)
add_executable(my_app, tu1.cpp, tu2.cpp, ..., tuN.cpp)
The build tool have to preprocess all the .cpp files, find all the
"#include" directives and generate the build rules:
# target dependencies
tu1.cpp.o : tu1.cpp, hA.hpp, ..., hX.hpp
...
tuN.cpp.o : tuN.cpp, hB.hpp, ..., hY.hpp
my_app : tu1.cpp.o, ..., tuN.cpp.o
Note that the *.o targets don't depend on each other and can be compiled
in-parallel.
--------------------------------------------
Modules add the module interface translation units (I'll call it an *.ixx
file) and compiled module exports (an *.ifc file)
If a translation unit A.cpp imports a module B, then it depends on B.ifc
which depends on B.ixx.
So if we have the following source files (for simplicity let's not use
header files)
a_tu1.ixx, a_tu2.cpp (imports B), ..., a_tuN.cpp // module A
b_tu1.ixx (imports A), b_tu2.cpp, ..., b_tuM.cpp // module B
...
m_tu1.ixx, m_tu2.cpp, ..., m_tuM.cpp // module M
This means that a build tool have to produce the following build rules:
# targets dependencies
a_tu1.ixx.o, A.ifc : a_tu1.ixx
b_tu1.ixx.o, B.ifc : b_tu1.ixx, A.ifc
...
m_tu1.ixx.o, M.ifc : m_tu1.ixx, X.ifc, ..., Y.ifc
a_tu2.cpp.o : a_tu2.cpp, B.ifc
...
m_tuN.cpp.o : m_tuN.cpp, X.ifc, ..., Y.ifc
my_app : a_tu1.ixx.o, ..., m_tuN.cpp.o
And now we have *.o targets which depend on *.ifc targets, and *.ifc
targets which depend on other *.ifc targets.
So what should we write in our build tool script, so it would generate such
build rules?
It would be great if we could just write down all the translation units:
add_executable(my_app, a_tu1.ixx.o, ..., a_tuN.cpp.o)
But then the build tool would have to not only preprocess C++ code (to find
all the #include directives), but also parse it, in order to find the
import statements.
It's not practical to manually write all the dependencies in the build
script, because this would just duplicate the import statements in the code:
add_module_dependencies(A, b_tu1.ixx, ..., x_tuX.cpp)
add_module_dependencies(B, a_tu2.cpp, ..., y_tuY.cpp)
-- or --
add_module_dependencies(b_tu1.ixx, A)
add_module_dependencies(a_tu2.cpp, B)
--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/a9ff0d91-dea2-49f1-8285-072732360d60%40isocpp.org.
------=_Part_100_446552970.1456500236420
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">Without modules we have the following situation:<br><br>Th=
ere is a bunch of source files with code -<br><br><div class=3D"prettyprint=
" style=3D"background-color: rgb(250, 250, 250); border-color: rgb(187, 187=
, 187); border-style: solid; border-width: 1px; word-wrap: break-word;"><co=
de class=3D"prettyprint"><div class=3D"subprettyprint"><span style=3D"color=
: #000;" class=3D"styled-by-prettify">=C2=A0 =C2=A0 tu1</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify">cpp</span><span style=3D"color: #660;=
" class=3D"styled-by-prettify">,</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify"> tu2</span><span style=3D"color: #660;" class=3D"st=
yled-by-prettify">.</span><span style=3D"color: #000;" class=3D"styled-by-p=
rettify">cpp</span><span style=3D"color: #660;" class=3D"styled-by-prettify=
">,</span><span style=3D"color: #000;" class=3D"styled-by-prettify"> </span=
><span style=3D"color: #660;" class=3D"styled-by-prettify">...,</span><span=
style=3D"color: #000;" class=3D"styled-by-prettify"> tuN</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify">cpp</span><span style=3D"color: #660;=
" class=3D"styled-by-prettify">,</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify"><br>=C2=A0 =C2=A0 h1</span><span style=3D"color: #6=
60;" class=3D"styled-by-prettify">.</span><span style=3D"color: #000;" clas=
s=3D"styled-by-prettify">hpp</span><span style=3D"color: #660;" class=3D"st=
yled-by-prettify">,</span><span style=3D"color: #000;" class=3D"styled-by-p=
rettify"> h2</span><span style=3D"color: #660;" class=3D"styled-by-prettify=
">.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">hpp</sp=
an><span style=3D"color: #660;" class=3D"styled-by-prettify">,</span><span =
style=3D"color: #000;" class=3D"styled-by-prettify"> </span><span style=3D"=
color: #660;" class=3D"styled-by-prettify">...,</span><span style=3D"color:=
#000;" class=3D"styled-by-prettify"> hM</span><span style=3D"color: #660;"=
class=3D"styled-by-prettify">.</span><span style=3D"color: #000;" class=3D=
"styled-by-prettify">hpp</span><span style=3D"color: #660;" class=3D"styled=
-by-prettify">,</span><span style=3D"color: #000;" class=3D"styled-by-prett=
ify"><br></span></div></code></div><br>And a build tool script (*make, gyp,=
gn, etc)<br><br><div class=3D"prettyprint" style=3D"background-color: rgb(=
250, 250, 250); border-color: rgb(187, 187, 187); border-style: solid; bord=
er-width: 1px; word-wrap: break-word;"><code class=3D"prettyprint"><div cla=
ss=3D"subprettyprint"><span style=3D"color: #000;" class=3D"styled-by-prett=
ify">=C2=A0 =C2=A0 add_executable</span><span style=3D"color: #660;" class=
=3D"styled-by-prettify">(</span><span style=3D"color: #000;" class=3D"style=
d-by-prettify">my_app</span><span style=3D"color: #660;" class=3D"styled-by=
-prettify">,</span><span style=3D"color: #000;" class=3D"styled-by-prettify=
"> tu1</span><span style=3D"color: #660;" class=3D"styled-by-prettify">.</s=
pan><span style=3D"color: #000;" class=3D"styled-by-prettify">cpp</span><sp=
an style=3D"color: #660;" class=3D"styled-by-prettify">,</span><span style=
=3D"color: #000;" class=3D"styled-by-prettify"> tu2</span><span style=3D"co=
lor: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color: #000=
;" class=3D"styled-by-prettify">cpp</span><span style=3D"color: #660;" clas=
s=3D"styled-by-prettify">,</span><span style=3D"color: #000;" class=3D"styl=
ed-by-prettify"> </span><span style=3D"color: #660;" class=3D"styled-by-pre=
ttify">...,</span><span style=3D"color: #000;" class=3D"styled-by-prettify"=
> tuN</span><span style=3D"color: #660;" class=3D"styled-by-prettify">.</sp=
an><span style=3D"color: #000;" class=3D"styled-by-prettify">cpp</span><spa=
n style=3D"color: #660;" class=3D"styled-by-prettify">)</span></div></code>=
</div><br>The build tool have to preprocess all the .cpp files, find all th=
e "#include" directives and generate the build rules:<br><br><div=
class=3D"prettyprint" style=3D"background-color: rgb(250, 250, 250); borde=
r-color: rgb(187, 187, 187); border-style: solid; border-width: 1px; word-w=
rap: break-word;"><code class=3D"prettyprint"><div class=3D"subprettyprint"=
><span style=3D"color: #000;" class=3D"styled-by-prettify">=C2=A0 =C2=A0 </=
span><span style=3D"color: #800;" class=3D"styled-by-prettify"># target =C2=
=A0 =C2=A0dependencies</span><span style=3D"color: #000;" class=3D"styled-b=
y-prettify"><br>=C2=A0 =C2=A0 tu1</span><span style=3D"color: #660;" class=
=3D"styled-by-prettify">.</span><span style=3D"color: #000;" class=3D"style=
d-by-prettify">cpp</span><span style=3D"color: #660;" class=3D"styled-by-pr=
ettify">.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">o=
</span><span style=3D"color: #660;" class=3D"styled-by-prettify">:</span><=
span style=3D"color: #000;" class=3D"styled-by-prettify"> tu1</span><span s=
tyle=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"c=
olor: #000;" class=3D"styled-by-prettify">cpp</span><span style=3D"color: #=
660;" class=3D"styled-by-prettify">,</span><span style=3D"color: #000;" cla=
ss=3D"styled-by-prettify"> hA</span><span style=3D"color: #660;" class=3D"s=
tyled-by-prettify">.</span><span style=3D"color: #000;" class=3D"styled-by-=
prettify">hpp</span><span style=3D"color: #660;" class=3D"styled-by-prettif=
y">,</span><span style=3D"color: #000;" class=3D"styled-by-prettify"> </spa=
n><span style=3D"color: #660;" class=3D"styled-by-prettify">...,</span><spa=
n style=3D"color: #000;" class=3D"styled-by-prettify"> hX</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify">hpp<br>=C2=A0 =C2=A0 </span><span sty=
le=3D"color: #660;" class=3D"styled-by-prettify">...</span><span style=3D"c=
olor: #000;" class=3D"styled-by-prettify"><br>=C2=A0 =C2=A0 tuN</span><span=
style=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D=
"color: #000;" class=3D"styled-by-prettify">cpp</span><span style=3D"color:=
#660;" class=3D"styled-by-prettify">.</span><span style=3D"color: #000;" c=
lass=3D"styled-by-prettify">o </span><span style=3D"color: #660;" class=3D"=
styled-by-prettify">:</span><span style=3D"color: #000;" class=3D"styled-by=
-prettify"> tuN</span><span style=3D"color: #660;" class=3D"styled-by-prett=
ify">.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">cpp<=
/span><span style=3D"color: #660;" class=3D"styled-by-prettify">,</span><sp=
an style=3D"color: #000;" class=3D"styled-by-prettify"> hB</span><span styl=
e=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"colo=
r: #000;" class=3D"styled-by-prettify">hpp</span><span style=3D"color: #660=
;" class=3D"styled-by-prettify">,</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify"> </span><span style=3D"color: #660;" class=3D"style=
d-by-prettify">...,</span><span style=3D"color: #000;" class=3D"styled-by-p=
rettify"> hY</span><span style=3D"color: #660;" class=3D"styled-by-prettify=
">.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">hpp<br>=
=C2=A0 =C2=A0 my_app =C2=A0 =C2=A0</span><span style=3D"color: #660;" class=
=3D"styled-by-prettify">:</span><span style=3D"color: #000;" class=3D"style=
d-by-prettify"> tu1</span><span style=3D"color: #660;" class=3D"styled-by-p=
rettify">.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">=
cpp</span><span style=3D"color: #660;" class=3D"styled-by-prettify">.</span=
><span style=3D"color: #000;" class=3D"styled-by-prettify">o</span><span st=
yle=3D"color: #660;" class=3D"styled-by-prettify">,</span><span style=3D"co=
lor: #000;" class=3D"styled-by-prettify"> </span><span style=3D"color: #660=
;" class=3D"styled-by-prettify">...,</span><span style=3D"color: #000;" cla=
ss=3D"styled-by-prettify"> tuN</span><span style=3D"color: #660;" class=3D"=
styled-by-prettify">.</span><span style=3D"color: #000;" class=3D"styled-by=
-prettify">cpp</span><span style=3D"color: #660;" class=3D"styled-by-pretti=
fy">.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">o<br>=
</span></div></code></div><br>Note that the *.o targets don't depend on=
each other and can be compiled in-parallel.<br><br>-----------------------=
---------------------<br><br>Modules add the module interface translation u=
nits (I'll call it an *.ixx file) and compiled module exports (an *.ifc=
file) <br>If a translation unit A.cpp imports a module B, then it depends =
on B.ifc which depends on B.ixx.<br><br>So if we have the following source =
files (for simplicity let's not use header files)<br><br><div class=3D"=
prettyprint" style=3D"background-color: rgb(250, 250, 250); border-color: r=
gb(187, 187, 187); border-style: solid; border-width: 1px; word-wrap: break=
-word;"><code class=3D"prettyprint"><div class=3D"subprettyprint"><span sty=
le=3D"color: #000;" class=3D"styled-by-prettify">=C2=A0 =C2=A0 a_tu1</span>=
<span style=3D"color: #660;" class=3D"styled-by-prettify">.</span><span sty=
le=3D"color: #000;" class=3D"styled-by-prettify">ixx</span><span style=3D"c=
olor: #660;" class=3D"styled-by-prettify">,</span><span style=3D"color: #00=
0;" class=3D"styled-by-prettify"> a_tu2</span><span style=3D"color: #660;" =
class=3D"styled-by-prettify">.</span><span style=3D"color: #000;" class=3D"=
styled-by-prettify">cpp </span><span style=3D"color: #660;" class=3D"styled=
-by-prettify">(</span><span style=3D"color: #000;" class=3D"styled-by-prett=
ify">imports B</span><span style=3D"color: #660;" class=3D"styled-by-pretti=
fy">),</span><span style=3D"color: #000;" class=3D"styled-by-prettify"> </s=
pan><span style=3D"color: #660;" class=3D"styled-by-prettify">...,</span><s=
pan style=3D"color: #000;" class=3D"styled-by-prettify"> a_tuN</span><span =
style=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"=
color: #000;" class=3D"styled-by-prettify">cpp </span><span style=3D"color:=
#800;" class=3D"styled-by-prettify">// module A</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify"><br>=C2=A0 =C2=A0 b_tu1</span><span s=
tyle=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"c=
olor: #000;" class=3D"styled-by-prettify">ixx </span><span style=3D"color: =
#660;" class=3D"styled-by-prettify">(</span><span style=3D"color: #000;" cl=
ass=3D"styled-by-prettify">imports A</span><span style=3D"color: #660;" cla=
ss=3D"styled-by-prettify">),</span><span style=3D"color: #000;" class=3D"st=
yled-by-prettify"> b_tu2</span><span style=3D"color: #660;" class=3D"styled=
-by-prettify">.</span><span style=3D"color: #000;" class=3D"styled-by-prett=
ify">cpp</span><span style=3D"color: #660;" class=3D"styled-by-prettify">,<=
/span><span style=3D"color: #000;" class=3D"styled-by-prettify"> </span><sp=
an style=3D"color: #660;" class=3D"styled-by-prettify">...,</span><span sty=
le=3D"color: #000;" class=3D"styled-by-prettify"> b_tuM</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify">cpp </span><span style=3D"color: #800=
;" class=3D"styled-by-prettify">// module B</span><span style=3D"color: #00=
0;" class=3D"styled-by-prettify"><br>=C2=A0 =C2=A0 </span><span style=3D"co=
lor: #660;" class=3D"styled-by-prettify">...</span><span style=3D"color: #0=
00;" class=3D"styled-by-prettify"><br>=C2=A0 =C2=A0 m_tu1</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify">ixx</span><span style=3D"color: #660;=
" class=3D"styled-by-prettify">,</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify"> m_tu2</span><span style=3D"color: #660;" class=3D"=
styled-by-prettify">.</span><span style=3D"color: #000;" class=3D"styled-by=
-prettify">cpp</span><span style=3D"color: #660;" class=3D"styled-by-pretti=
fy">,</span><span style=3D"color: #000;" class=3D"styled-by-prettify"> </sp=
an><span style=3D"color: #660;" class=3D"styled-by-prettify">...,</span><sp=
an style=3D"color: #000;" class=3D"styled-by-prettify"> m_tuM</span><span s=
tyle=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"c=
olor: #000;" class=3D"styled-by-prettify">cpp </span><span style=3D"color: =
#800;" class=3D"styled-by-prettify">// module M</span></div></code></div><b=
r>This means that a build tool have to produce the following build rules:<b=
r><br><div class=3D"prettyprint" style=3D"background-color: rgb(250, 250, 2=
50); border-color: rgb(187, 187, 187); border-style: solid; border-width: 1=
px; word-wrap: break-word;"><code class=3D"prettyprint"><div class=3D"subpr=
ettyprint"><span style=3D"color: #000;" class=3D"styled-by-prettify">=C2=A0=
=C2=A0 </span><span style=3D"color: #800;" class=3D"styled-by-prettify"># =
targets =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 dependencies</span><span =
style=3D"color: #000;" class=3D"styled-by-prettify"><br>=C2=A0 =C2=A0 a_tu1=
</span><span style=3D"color: #660;" class=3D"styled-by-prettify">.</span><s=
pan style=3D"color: #000;" class=3D"styled-by-prettify">ixx</span><span sty=
le=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"col=
or: #000;" class=3D"styled-by-prettify">o</span><span style=3D"color: #660;=
" class=3D"styled-by-prettify">,</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify"> A</span><span style=3D"color: #660;" class=3D"styl=
ed-by-prettify">.</span><span style=3D"color: #000;" class=3D"styled-by-pre=
ttify">ifc =C2=A0</span><span style=3D"color: #660;" class=3D"styled-by-pre=
ttify">:</span><span style=3D"color: #000;" class=3D"styled-by-prettify"> a=
_tu1</span><span style=3D"color: #660;" class=3D"styled-by-prettify">.</spa=
n><span style=3D"color: #000;" class=3D"styled-by-prettify">ixx<br>=C2=A0 =
=C2=A0 b_tu1</span><span style=3D"color: #660;" class=3D"styled-by-prettify=
">.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">ixx</sp=
an><span style=3D"color: #660;" class=3D"styled-by-prettify">.</span><span =
style=3D"color: #000;" class=3D"styled-by-prettify">o</span><span style=3D"=
color: #660;" class=3D"styled-by-prettify">,</span><span style=3D"color: #0=
00;" class=3D"styled-by-prettify"> B</span><span style=3D"color: #660;" cla=
ss=3D"styled-by-prettify">.</span><span style=3D"color: #000;" class=3D"sty=
led-by-prettify">ifc =C2=A0</span><span style=3D"color: #660;" class=3D"sty=
led-by-prettify">:</span><span style=3D"color: #000;" class=3D"styled-by-pr=
ettify"> b_tu1</span><span style=3D"color: #660;" class=3D"styled-by-pretti=
fy">.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">ixx</=
span><span style=3D"color: #660;" class=3D"styled-by-prettify">,</span><spa=
n style=3D"color: #000;" class=3D"styled-by-prettify"> A</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify">ifc<br>=C2=A0 =C2=A0 </span><span sty=
le=3D"color: #660;" class=3D"styled-by-prettify">...</span><span style=3D"c=
olor: #000;" class=3D"styled-by-prettify"><br>=C2=A0 =C2=A0 m_tu1</span><sp=
an style=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=
=3D"color: #000;" class=3D"styled-by-prettify">ixx</span><span style=3D"col=
or: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color: #000;=
" class=3D"styled-by-prettify">o</span><span style=3D"color: #660;" class=
=3D"styled-by-prettify">,</span><span style=3D"color: #000;" class=3D"style=
d-by-prettify"> M</span><span style=3D"color: #660;" class=3D"styled-by-pre=
ttify">.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">if=
c =C2=A0</span><span style=3D"color: #660;" class=3D"styled-by-prettify">:<=
/span><span style=3D"color: #000;" class=3D"styled-by-prettify"> m_tu1</spa=
n><span style=3D"color: #660;" class=3D"styled-by-prettify">.</span><span s=
tyle=3D"color: #000;" class=3D"styled-by-prettify">ixx</span><span style=3D=
"color: #660;" class=3D"styled-by-prettify">,</span><span style=3D"color: #=
000;" class=3D"styled-by-prettify"> X</span><span style=3D"color: #660;" cl=
ass=3D"styled-by-prettify">.</span><span style=3D"color: #000;" class=3D"st=
yled-by-prettify">ifc</span><span style=3D"color: #660;" class=3D"styled-by=
-prettify">,</span><span style=3D"color: #000;" class=3D"styled-by-prettify=
"> </span><span style=3D"color: #660;" class=3D"styled-by-prettify">...,</s=
pan><span style=3D"color: #000;" class=3D"styled-by-prettify"> Y</span><spa=
n style=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=
=3D"color: #000;" class=3D"styled-by-prettify">ifc<br>=C2=A0 =C2=A0 a_tu2</=
span><span style=3D"color: #660;" class=3D"styled-by-prettify">.</span><spa=
n style=3D"color: #000;" class=3D"styled-by-prettify">cpp</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify">o =C2=A0 =C2=A0 =C2=A0 =C2=A0 </span>=
<span style=3D"color: #660;" class=3D"styled-by-prettify">:</span><span sty=
le=3D"color: #000;" class=3D"styled-by-prettify"> a_tu2</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify">cpp</span><span style=3D"color: #660;=
" class=3D"styled-by-prettify">,</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify"> B</span><span style=3D"color: #660;" class=3D"styl=
ed-by-prettify">.</span><span style=3D"color: #000;" class=3D"styled-by-pre=
ttify">ifc<br>=C2=A0 =C2=A0 </span><span style=3D"color: #660;" class=3D"st=
yled-by-prettify">...</span><span style=3D"color: #000;" class=3D"styled-by=
-prettify"><br>=C2=A0 =C2=A0 m_tuN</span><span style=3D"color: #660;" class=
=3D"styled-by-prettify">.</span><span style=3D"color: #000;" class=3D"style=
d-by-prettify">cpp</span><span style=3D"color: #660;" class=3D"styled-by-pr=
ettify">.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">o=
=C2=A0 =C2=A0 =C2=A0 =C2=A0 </span><span style=3D"color: #660;" class=3D"s=
tyled-by-prettify">:</span><span style=3D"color: #000;" class=3D"styled-by-=
prettify"> m_tuN</span><span style=3D"color: #660;" class=3D"styled-by-pret=
tify">.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">cpp=
</span><span style=3D"color: #660;" class=3D"styled-by-prettify">,</span><s=
pan style=3D"color: #000;" class=3D"styled-by-prettify"> X</span><span styl=
e=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"colo=
r: #000;" class=3D"styled-by-prettify">ifc</span><span style=3D"color: #660=
;" class=3D"styled-by-prettify">,</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify"> </span><span style=3D"color: #660;" class=3D"style=
d-by-prettify">...,</span><span style=3D"color: #000;" class=3D"styled-by-p=
rettify"> Y</span><span style=3D"color: #660;" class=3D"styled-by-prettify"=
>.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">ifc<br>=
=C2=A0 =C2=A0 my_app =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0</span=
><span style=3D"color: #660;" class=3D"styled-by-prettify">:</span><span st=
yle=3D"color: #000;" class=3D"styled-by-prettify"> a_tu1</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify">ixx</span><span style=3D"color: #660;=
" class=3D"styled-by-prettify">.</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify">o</span><span style=3D"color: #660;" class=3D"style=
d-by-prettify">,</span><span style=3D"color: #000;" class=3D"styled-by-pret=
tify"> </span><span style=3D"color: #660;" class=3D"styled-by-prettify">...=
,</span><span style=3D"color: #000;" class=3D"styled-by-prettify"> m_tuN</s=
pan><span style=3D"color: #660;" class=3D"styled-by-prettify">.</span><span=
style=3D"color: #000;" class=3D"styled-by-prettify">cpp</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify">o<br></span></div></code></div><br>An=
d now we have *.o targets which depend on *.ifc targets, and *.ifc targets =
which depend on other *.ifc targets.<br><br>So what should we write in our =
build tool script, so it would generate such build rules?<br><br>It would b=
e great if we could just write down all the translation units:<br><br><div =
class=3D"prettyprint" style=3D"background-color: rgb(250, 250, 250); border=
-color: rgb(187, 187, 187); border-style: solid; border-width: 1px; word-wr=
ap: break-word;"><code class=3D"prettyprint"><div class=3D"subprettyprint">=
<span style=3D"color: #000;" class=3D"styled-by-prettify">=C2=A0 =C2=A0 add=
_executable</span><span style=3D"color: #660;" class=3D"styled-by-prettify"=
>(</span><span style=3D"color: #000;" class=3D"styled-by-prettify">my_app</=
span><span style=3D"color: #660;" class=3D"styled-by-prettify">,</span><spa=
n style=3D"color: #000;" class=3D"styled-by-prettify"> a_tu1</span><span st=
yle=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"co=
lor: #000;" class=3D"styled-by-prettify">ixx</span><span style=3D"color: #6=
60;" class=3D"styled-by-prettify">.</span><span style=3D"color: #000;" clas=
s=3D"styled-by-prettify">o</span><span style=3D"color: #660;" class=3D"styl=
ed-by-prettify">,</span><span style=3D"color: #000;" class=3D"styled-by-pre=
ttify"> </span><span style=3D"color: #660;" class=3D"styled-by-prettify">..=
..,</span><span style=3D"color: #000;" class=3D"styled-by-prettify"> a_tuN</=
span><span style=3D"color: #660;" class=3D"styled-by-prettify">.</span><spa=
n style=3D"color: #000;" class=3D"styled-by-prettify">cpp</span><span style=
=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color=
: #000;" class=3D"styled-by-prettify">o</span><span style=3D"color: #660;" =
class=3D"styled-by-prettify">)</span></div></code></div><br>But then the bu=
ild tool would have to not only preprocess C++ code (to find all the #inclu=
de directives), but also parse it, in order to find the import statements.<=
br><br>It's not practical to manually write all the dependencies in the=
build script, because this would just duplicate the import statements in t=
he code:<br><br><div class=3D"prettyprint" style=3D"background-color: rgb(2=
50, 250, 250); border-color: rgb(187, 187, 187); border-style: solid; borde=
r-width: 1px; word-wrap: break-word;"><code class=3D"prettyprint"><div clas=
s=3D"subprettyprint"><span style=3D"color: #000;" class=3D"styled-by-pretti=
fy">=C2=A0 =C2=A0 add_module_dependencies</span><span style=3D"color: #660;=
" class=3D"styled-by-prettify">(</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify">A</span><span style=3D"color: #660;" class=3D"style=
d-by-prettify">,</span><span style=3D"color: #000;" class=3D"styled-by-pret=
tify"> b_tu1</span><span style=3D"color: #660;" class=3D"styled-by-prettify=
">.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">ixx</sp=
an><span style=3D"color: #660;" class=3D"styled-by-prettify">,</span><span =
style=3D"color: #000;" class=3D"styled-by-prettify"> </span><span style=3D"=
color: #660;" class=3D"styled-by-prettify">...,</span><span style=3D"color:=
#000;" class=3D"styled-by-prettify"> x_tuX</span><span style=3D"color: #66=
0;" class=3D"styled-by-prettify">.</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify">cpp</span><span style=3D"color: #660;" class=3D"sty=
led-by-prettify">)</span><span style=3D"color: #000;" class=3D"styled-by-pr=
ettify"><br>=C2=A0 =C2=A0 add_module_dependencies</span><span style=3D"colo=
r: #660;" class=3D"styled-by-prettify">(</span><span style=3D"color: #000;"=
class=3D"styled-by-prettify">B</span><span style=3D"color: #660;" class=3D=
"styled-by-prettify">,</span><span style=3D"color: #000;" class=3D"styled-b=
y-prettify"> a_tu2</span><span style=3D"color: #660;" class=3D"styled-by-pr=
ettify">.</span><span style=3D"color: #000;" class=3D"styled-by-prettify">c=
pp</span><span style=3D"color: #660;" class=3D"styled-by-prettify">,</span>=
<span style=3D"color: #000;" class=3D"styled-by-prettify"> </span><span sty=
le=3D"color: #660;" class=3D"styled-by-prettify">...,</span><span style=3D"=
color: #000;" class=3D"styled-by-prettify"> y_tuY</span><span style=3D"colo=
r: #660;" class=3D"styled-by-prettify">.</span><span style=3D"color: #000;"=
class=3D"styled-by-prettify">cpp</span><span style=3D"color: #660;" class=
=3D"styled-by-prettify">)</span><span style=3D"color: #000;" class=3D"style=
d-by-prettify"><br>=C2=A0 =C2=A0 </span><span style=3D"color: #660;" class=
=3D"styled-by-prettify">--</span><span style=3D"color: #000;" class=3D"styl=
ed-by-prettify"> </span><span style=3D"color: #008;" class=3D"styled-by-pre=
ttify">or</span><span style=3D"color: #000;" class=3D"styled-by-prettify"> =
</span><span style=3D"color: #660;" class=3D"styled-by-prettify">--</span><=
span style=3D"color: #000;" class=3D"styled-by-prettify"><br>=C2=A0 =C2=A0 =
add_module_dependencies</span><span style=3D"color: #660;" class=3D"styled-=
by-prettify">(</span><span style=3D"color: #000;" class=3D"styled-by-pretti=
fy">b_tu1</span><span style=3D"color: #660;" class=3D"styled-by-prettify">.=
</span><span style=3D"color: #000;" class=3D"styled-by-prettify">ixx</span>=
<span style=3D"color: #660;" class=3D"styled-by-prettify">,</span><span sty=
le=3D"color: #000;" class=3D"styled-by-prettify"> A</span><span style=3D"co=
lor: #660;" class=3D"styled-by-prettify">)</span><span style=3D"color: #000=
;" class=3D"styled-by-prettify"><br>=C2=A0 =C2=A0 add_module_dependencies</=
span><span style=3D"color: #660;" class=3D"styled-by-prettify">(</span><spa=
n style=3D"color: #000;" class=3D"styled-by-prettify">a_tu2</span><span sty=
le=3D"color: #660;" class=3D"styled-by-prettify">.</span><span style=3D"col=
or: #000;" class=3D"styled-by-prettify">cpp</span><span style=3D"color: #66=
0;" class=3D"styled-by-prettify">,</span><span style=3D"color: #000;" class=
=3D"styled-by-prettify"> B</span><span style=3D"color: #660;" class=3D"styl=
ed-by-prettify">)</span><span style=3D"color: #000;" class=3D"styled-by-pre=
ttify"><br><br></span></div></code></div><br></div>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;ISO C++ Standard - Future Proposals" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:std-proposals+unsubscribe@isocpp.org">std-proposa=
ls+unsubscribe@isocpp.org</a>.<br />
To post to this group, send email to <a href=3D"mailto:std-proposals@isocpp=
..org">std-proposals@isocpp.org</a>.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/a/isocpp.org/d/msgid/std-proposals/a9ff0d91-dea2-49f1-8285-072732360d60%=
40isocpp.org?utm_medium=3Demail&utm_source=3Dfooter">https://groups.google.=
com/a/isocpp.org/d/msgid/std-proposals/a9ff0d91-dea2-49f1-8285-072732360d60=
%40isocpp.org</a>.<br />
------=_Part_100_446552970.1456500236420--
------=_Part_99_1784697087.1456500236418--
.
Author: Richard Smith <richard@metafoo.co.uk>
Date: Fri, 26 Feb 2016 11:57:40 -0800
Raw View
On Fri, Feb 26, 2016 at 7:23 AM, Abyx <fl.lllj@gmail.com> wrote:
> Without modules we have the following situation:
>
> There is a bunch of source files with code -
>
> tu1.cpp, tu2.cpp, ..., tuN.cpp,
> h1.hpp, h2.hpp, ..., hM.hpp,
>
> And a build tool script (*make, gyp, gn, etc)
>
> add_executable(my_app, tu1.cpp, tu2.cpp, ..., tuN.cpp)
>
> The build tool have to preprocess all the .cpp files, find all the
> "#include" directives and generate the build rules:
>
> # target dependencies
> tu1.cpp.o : tu1.cpp, hA.hpp, ..., hX.hpp
> ...
> tuN.cpp.o : tuN.cpp, hB.hpp, ..., hY.hpp
> my_app : tu1.cpp.o, ..., tuN.cpp.o
>
> Note that the *.o targets don't depend on each other and can be compiled
> in-parallel.
>
> --------------------------------------------
>
> Modules add the module interface translation units (I'll call it an *.ixx
> file) and compiled module exports (an *.ifc file)
> If a translation unit A.cpp imports a module B, then it depends on B.ifc
> which depends on B.ixx.
>
> So if we have the following source files (for simplicity let's not use
> header files)
>
> a_tu1.ixx, a_tu2.cpp (imports B), ..., a_tuN.cpp // module A
> b_tu1.ixx (imports A), b_tu2.cpp, ..., b_tuM.cpp // module B
> ...
> m_tu1.ixx, m_tu2.cpp, ..., m_tuM.cpp // module M
>
> This means that a build tool have to produce the following build rules:
>
> # targets dependencies
> a_tu1.ixx.o, A.ifc : a_tu1.ixx
> b_tu1.ixx.o, B.ifc : b_tu1.ixx, A.ifc
> ...
> m_tu1.ixx.o, M.ifc : m_tu1.ixx, X.ifc, ..., Y.ifc
> a_tu2.cpp.o : a_tu2.cpp, B.ifc
> ...
> m_tuN.cpp.o : m_tuN.cpp, X.ifc, ..., Y.ifc
> my_app : a_tu1.ixx.o, ..., m_tuN.cpp.o
>
> And now we have *.o targets which depend on *.ifc targets, and *.ifc targets
> which depend on other *.ifc targets.
>
> So what should we write in our build tool script, so it would generate such
> build rules?
>
> It would be great if we could just write down all the translation units:
>
> add_executable(my_app, a_tu1.ixx.o, ..., a_tuN.cpp.o)
>
> But then the build tool would have to not only preprocess C++ code (to find
> all the #include directives), but also parse it, in order to find the import
> statements.
>
> It's not practical to manually write all the dependencies in the build
> script, because this would just duplicate the import statements in the code:
>
> add_module_dependencies(A, b_tu1.ixx, ..., x_tuX.cpp)
> add_module_dependencies(B, a_tu2.cpp, ..., y_tuY.cpp)
> -- or --
> add_module_dependencies(b_tu1.ixx, A)
> add_module_dependencies(a_tu2.cpp, B)
You have (at least) three choices:
1) You manually list the dependencies between your libraries in your
build rules. This is often a good thing for your code health, but may
not be right for everyone.
2) Your build process scans for import statements. This is not really
any harder than scanning for #includes; no real parsing is required,
at most, you need tokenize and maybe preprocess (if you want 'import
MACRO_NAME' to work). The same is true for #includes.
3) You ask your compiler to implicitly build (and cache) module
interfaces on demand. This requires that your compiler has some way to
map from an imported module name to the relevant interface file(s).
That could happen via some implementation-defined means (such as
Clang's module map files) or by making the module names directly
correspond to module interface files (as suggested in
http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0273r0.pdf).
--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/CAOfiQqnM09zTWXR_0NDj3f%3DQ1mqoO66hNeg5SPEPU9MX0eM_Ng%40mail.gmail.com.
.
Author: Sean Middleditch <sean.middleditch@gmail.com>
Date: Sun, 28 Feb 2016 17:41:06 -0800 (PST)
Raw View
------=_Part_101_638399159.1456710066362
Content-Type: multipart/alternative;
boundary="----=_Part_102_513709644.1456710066363"
------=_Part_102_513709644.1456710066363
Content-Type: text/plain; charset=UTF-8
I'll start off by saying that I don't think that Abyx's worries are a
significant problem. Yes, build systems will need to change, but they've
needed to change for 20 years now. If modules are the impetus for the
community to finally make building C++ not ridiculous, that's just fine
with me. I'm actually really hoping that modules are the final catalyst for
the community to realize that CMake is this generations' Autotools. ;)
That said, I do think it's worthwhile that folks at least understand where
Abyx is coming from.
On Friday, February 26, 2016 at 11:57:44 AM UTC-8, Richard Smith wrote:
> On Fri, Feb 26, 2016 at 7:23 AM, Abyx <fl....@gmail.com <javascript:>>
> wrote:
>
1) You manually list the dependencies between your libraries in your
build rules. This is often a good thing for your code health, but may
not be right for everyone.
We're not talking about libraries! We're talking about translation units
and modules.
A single library could be made up of dozens and dozens (if not hundreds) of
TUs and modules.
> 2) Your build process scans for import statements. This is not really
> any harder than scanning for #includes; no real parsing is required,
> at most, you need tokenize and maybe preprocess (if you want 'import
> MACRO_NAME' to work). The same is true for #includes.
This is much harder than scanning for #include files. No good modern C++
build chain actually has a separate "scan for includes" step anymore. :)
With headers, the dependency collection is just part of building. MSC, GCC,
etc. all have flags that allow them to generate dependency chains while
compiling, e.g. -MM -MF <file> on GCC. The order of compilation of TUs is
usually irrelevant; in fact, it's safe to build a dependent without even
knowing the dependency exists, so long as all the dependee files already
exist. The dependencies are used only for checking whether files are out of
date relative to another file.
In fact, A.cpp and B.cpp can be compiled in any order, or compiled in
parallel, or compiled on two different machines, even if B.cpp has a
dependency on A.h.
With modules, this is no longer the case. The dependency graph must be
present before any compilation begins, as building properly is now
dependent on per-TU build artifacts (the .ifc files). Distributed builds
require synchronization of the artifacts or island solving to ensure that
files with dependency chains are all built on the same build node. This is
because B.cpp no longer would depend on A.h, but instead would depend on
A.ifc, which doesn't even exist until after A.cpp is compiled.
Note in particular that this also means that with modules we've just
serialized a previously parallel process, which is theoretically a build
deoptimization - seemingly the exact opposite of what many of us are
expecting or want out of modules!
That said, your option 3 is the obvious solution. It's roughly what most
other modern languages with module systems do, and C++ implementations and
the build tools we use are going to have to modernize to adopt to a world
where the compiler plays a more integral role in the build chain than it
did before.
>
> 3) You ask your compiler to implicitly build (and cache) module
> interfaces on demand. This requires that your compiler has some way to
> map from an imported module name to the relevant interface file(s).
> That could happen via some implementation-defined means (such as
> Clang's module map files) or by making the module names directly
> correspond to module interface files (as suggested in
> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0273r0.pdf).
--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/d30abb01-3e49-424a-a410-556b60b1b833%40isocpp.org.
------=_Part_102_513709644.1456710066363
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><div>I'll start off by saying that I don't think t=
hat Abyx's worries are a significant problem. Yes, build systems will n=
eed to change, but they've needed to change for 20 years now. If module=
s are the impetus for the community to finally make building C++ not ridicu=
lous, that's just fine with me. I'm actually really hoping that mod=
ules are the final catalyst for the community to realize that CMake is this=
generations' Autotools. ;)</div><div><br></div><div>That said, I do th=
ink it's worthwhile that folks at least understand where Abyx is coming=
from.<br></div><div><br></div><div>On Friday, February 26, 2016 at 11:57:4=
4 AM UTC-8, Richard Smith wrote:<br></div><blockquote class=3D"gmail_quote"=
style=3D"margin: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;padding-=
left: 1ex;">On Fri, Feb 26, 2016 at 7:23 AM, Abyx <<a href=3D"javascript=
:" target=3D"_blank" gdf-obfuscated-mailto=3D"zA11OVz7AgAJ" rel=3D"nofollow=
" onmousedown=3D"this.href=3D'javascript:';return true;" onclick=3D=
"this.href=3D'javascript:';return true;">fl....@gmail.com</a>> w=
rote:
<br></blockquote><div><br></div><blockquote class=3D"gmail_quote" style=3D"=
margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-color: rgb(2=
04, 204, 204); border-left-style: solid; padding-left: 1ex;">1) You manuall=
y list the dependencies between your libraries in your=C2=A0</blockquote><b=
lockquote class=3D"gmail_quote" style=3D"margin: 0px 0px 0px 0.8ex; border-=
left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: =
solid; padding-left: 1ex;">build rules. This is often a good thing for your=
code health, but may=C2=A0</blockquote><blockquote class=3D"gmail_quote" s=
tyle=3D"margin: 0px 0px 0px 0.8ex; border-left-width: 1px; border-left-colo=
r: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex;">not be=
right for everyone.=C2=A0=C2=A0</blockquote><div><br></div><div>We're =
not talking about libraries! We're talking about translation units and =
modules.</div><div><br></div><div>A single library could be made up of doze=
ns and dozens (if not hundreds) of TUs and modules.</div><div>=C2=A0</div><=
blockquote class=3D"gmail_quote" style=3D"margin: 0;margin-left: 0.8ex;bord=
er-left: 1px #ccc solid;padding-left: 1ex;">2) Your build process scans for=
import statements. This is not really
<br>any harder than scanning for #includes; no real parsing is required,
<br>at most, you need tokenize and maybe preprocess (if you want 'impor=
t
<br>MACRO_NAME' to work). The same is true for #includes. =C2=A0</block=
quote><div><br></div><div>This is much harder than scanning for #include fi=
les. No good modern C++ build chain actually has a separate "scan for =
includes" step anymore. :)<br></div><div><br></div><div>With headers, =
the dependency collection is just part of building. MSC, GCC, etc. all have=
flags that allow them to generate dependency chains while compiling, e.g. =
-MM -MF <file> on GCC. The order of compilation of TUs is usually irr=
elevant; in fact, it's safe to build a dependent without even knowing t=
he dependency exists, so long as all the dependee files already exist. The =
dependencies are used only for checking whether files are out of date relat=
ive to another file.</div><div><br></div><div>In fact, A.cpp and B.cpp can =
be compiled in any order, or compiled in parallel, or compiled on two diffe=
rent machines, even if B.cpp has a dependency on A.h.</div><div><br></div><=
div>With modules, this is no longer the case. The dependency graph must be =
present before any compilation begins, as building properly is now dependen=
t on per-TU build artifacts (the .ifc files). Distributed builds require sy=
nchronization of the artifacts or island solving to ensure that files with =
dependency chains are all built on the same build node. This is because B.c=
pp no longer would depend on A.h, but instead would depend on A.ifc, which =
doesn't even exist until after A.cpp is compiled.<br></div><div><br></d=
iv><div>Note in particular that this also means that with modules we've=
just serialized a previously parallel process, which is theoretically a bu=
ild deoptimization - seemingly the exact opposite of what many of us are ex=
pecting or want out of modules!</div><div><br></div><div>That said, your op=
tion 3 is the obvious solution. It's roughly what most other modern lan=
guages with module systems do, and C++ implementations and the build tools =
we use are going to have to modernize to adopt to a world where the compile=
r plays a more integral role in the build chain than it did before.</div><d=
iv>=C2=A0<br></div><blockquote class=3D"gmail_quote" style=3D"margin: 0;mar=
gin-left: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;">
<br>3) You ask your compiler to implicitly build (and cache) module
<br>interfaces on demand. This requires that your compiler has some way to
<br>map from an imported module name to the relevant interface file(s).
<br>That could happen via some implementation-defined means (such as
<br>Clang's module map files) or by making the module names directly
<br>correspond to module interface files (as suggested in
<br><a href=3D"http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p027=
3r0.pdf" target=3D"_blank" rel=3D"nofollow" onmousedown=3D"this.href=3D'=
;http://www.google.com/url?q\75http%3A%2F%2Fwww.open-std.org%2Fjtc1%2Fsc22%=
2Fwg21%2Fdocs%2Fpapers%2F2016%2Fp0273r0.pdf\46sa\75D\46sntz\0751\46usg\75AF=
QjCNFuSHtll19rhWabB9b_WnoR8Q0W_w';return true;" onclick=3D"this.href=3D=
'http://www.google.com/url?q\75http%3A%2F%2Fwww.open-std.org%2Fjtc1%2Fs=
c22%2Fwg21%2Fdocs%2Fpapers%2F2016%2Fp0273r0.pdf\46sa\75D\46sntz\0751\46usg\=
75AFQjCNFuSHtll19rhWabB9b_WnoR8Q0W_w';return true;">http://www.open-std=
..org/jtc1/<wbr>sc22/wg21/docs/papers/2016/<wbr>p0273r0.pdf</a>).=C2=A0</blo=
ckquote></div>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;ISO C++ Standard - Future Proposals" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:std-proposals+unsubscribe@isocpp.org">std-proposa=
ls+unsubscribe@isocpp.org</a>.<br />
To post to this group, send email to <a href=3D"mailto:std-proposals@isocpp=
..org">std-proposals@isocpp.org</a>.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/a/isocpp.org/d/msgid/std-proposals/d30abb01-3e49-424a-a410-556b60b1b833%=
40isocpp.org?utm_medium=3Demail&utm_source=3Dfooter">https://groups.google.=
com/a/isocpp.org/d/msgid/std-proposals/d30abb01-3e49-424a-a410-556b60b1b833=
%40isocpp.org</a>.<br />
------=_Part_102_513709644.1456710066363--
------=_Part_101_638399159.1456710066362--
.
Author: FrankHB1989 <frankhb1989@gmail.com>
Date: Sun, 28 Feb 2016 22:42:41 -0800 (PST)
Raw View
------=_Part_792_197808656.1456728161680
Content-Type: multipart/alternative;
boundary="----=_Part_793_1406381613.1456728161680"
------=_Part_793_1406381613.1456728161680
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
=E5=9C=A8 2016=E5=B9=B42=E6=9C=8829=E6=97=A5=E6=98=9F=E6=9C=9F=E4=B8=80 UTC=
+8=E4=B8=8A=E5=8D=889:41:06=EF=BC=8CSean Middleditch=E5=86=99=E9=81=93=EF=
=BC=9A
>
> I'll start off by saying that I don't think that Abyx's worries are a=20
> significant problem. Yes, build systems will need to change, but they've=
=20
> needed to change for 20 years now. If modules are the impetus for the=20
> community to finally make building C++ not ridiculous, that's just fine=
=20
> with me. I'm actually really hoping that modules are the final catalyst f=
or=20
> the community to realize that CMake is this generations' Autotools. ;)
>
> That said, I do think it's worthwhile that folks at least understand wher=
e=20
> Abyx is coming from.
>
> On Friday, February 26, 2016 at 11:57:44 AM UTC-8, Richard Smith wrote:
>
>> On Fri, Feb 26, 2016 at 7:23 AM, Abyx <fl....@gmail.com> wrote:=20
>>
>
> 1) You manually list the dependencies between your libraries in your=20
>
> build rules. This is often a good thing for your code health, but may=20
>
> not be right for everyone. =20
>
>
> We're not talking about libraries! We're talking about translation units=
=20
> and modules.
>
> A single library could be made up of dozens and dozens (if not hundreds)=
=20
> of TUs and modules.
> =20
>
>> 2) Your build process scans for import statements. This is not really=20
>> any harder than scanning for #includes; no real parsing is required,=20
>> at most, you need tokenize and maybe preprocess (if you want 'import=20
>> MACRO_NAME' to work). The same is true for #includes. =20
>
>
> This is much harder than scanning for #include files. No good modern C++=
=20
> build chain actually has a separate "scan for includes" step anymore. :)
>
Do you mean scan by searching the text? Otherwise, the compiler may do some=
=20
"scan" work (as you said below).
It certainly should not be the right way. However, there are tools do that=
=20
in practice, e.g. Code::Blocks. (yep, not so modern & good...)=20
=20
>
> With headers, the dependency collection is just part of building. MSC,=20
> GCC, etc. all have flags that allow them to generate dependency chains=20
> while compiling, e.g. -MM -MF <file> on GCC. The order of compilation of=
=20
> TUs is usually irrelevant; in fact, it's safe to build a dependent withou=
t=20
> even knowing the dependency exists, so long as all the dependee files=20
> already exist. The dependencies are used only for checking whether files=
=20
> are out of date relative to another file.
>
> In fact, A.cpp and B.cpp can be compiled in any order, or compiled in=20
> parallel, or compiled on two different machines, even if B.cpp has a=20
> dependency on A.h.
>
> Agreed. And g++ -MMD as a single step works as well, though the result=20
(makefile deps) may need additional parsing process, but not hard. I've=20
actually implemented it in several lines=20
<https://bitbucket.org/FrankHB/yslib/src/2824dc90d869be7246b14abb6332e1997d=
d34a63/YFramework/source/NPL/Dependency.cpp?at=3Dmaster&fileviewer=3Dfile-v=
iew-default#Dependency.cpp-39>=20
with aid of my parsing library utilities and integrated it into my own=20
implemented-in-one-file build tool=20
<https://bitbucket.org/FrankHB/yslib/src/2824dc90d869be7246b14abb6332e1997d=
d34a63/Tools/SHBuild/Main.cpp?at=3Dmaster&fileviewer=3Dfile-view-default#Ma=
in.cpp-463>
..
=20
> With modules, this is no longer the case. The dependency graph must be=20
> present before any compilation begins, as building properly is now=20
> dependent on per-TU build artifacts (the .ifc files). Distributed builds=
=20
> require synchronization of the artifacts or island solving to ensure that=
=20
> files with dependency chains are all built on the same build node. This i=
s=20
> because B.cpp no longer would depend on A.h, but instead would depend on=
=20
> A.ifc, which doesn't even exist until after A.cpp is compiled.
>
> Note in particular that this also means that with modules we've just=20
> serialized a previously parallel process, which is theoretically a build=
=20
> deoptimization - seemingly the exact opposite of what many of us are=20
> expecting or want out of modules!
>
> That said, your option 3 is the obvious solution. It's roughly what most=
=20
> other modern languages with module systems do, and C++ implementations an=
d=20
> the build tools we use are going to have to modernize to adopt to a world=
=20
> where the compiler plays a more integral role in the build chain than it=
=20
> did before.
> =20
>
>>
>> 3) You ask your compiler to implicitly build (and cache) module=20
>> interfaces on demand. This requires that your compiler has some way to=
=20
>> map from an imported module name to the relevant interface file(s).=20
>> That could happen via some implementation-defined means (such as=20
>> Clang's module map files) or by making the module names directly=20
>> correspond to module interface files (as suggested in=20
>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0273r0.pdf=20
>> <http://www.google.com/url?q=3Dhttp%3A%2F%2Fwww.open-std.org%2Fjtc1%2Fsc=
22%2Fwg21%2Fdocs%2Fpapers%2F2016%2Fp0273r0.pdf&sa=3DD&sntz=3D1&usg=3DAFQjCN=
FuSHtll19rhWabB9b_WnoR8Q0W_w>
>> ).=20
>
>
I'm worrying there are many details not clear to average C++ users. And=20
comparing to the complexity of the specification, the whole system still=20
seems to be too weak in functionality. Are there anything like *structures*=
,=20
*signatures* and *functors* in module systems of ML-like languages proposed=
?
=20
--=20
You received this message because you are subscribed to the Google Groups "=
ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp=
..org/d/msgid/std-proposals/29edd7b0-671f-49ea-8538-46070981228f%40isocpp.or=
g.
------=_Part_793_1406381613.1456728161680
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr"><br><br>=E5=9C=A8 2016=E5=B9=B42=E6=9C=8829=E6=97=A5=E6=98=
=9F=E6=9C=9F=E4=B8=80 UTC+8=E4=B8=8A=E5=8D=889:41:06=EF=BC=8CSean Middledit=
ch=E5=86=99=E9=81=93=EF=BC=9A<blockquote class=3D"gmail_quote" style=3D"mar=
gin: 0;margin-left: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;"><=
div dir=3D"ltr"><div>I'll start off by saying that I don't think th=
at Abyx's worries are a significant problem. Yes, build systems will ne=
ed to change, but they've needed to change for 20 years now. If modules=
are the impetus for the community to finally make building C++ not ridicul=
ous, that's just fine with me. I'm actually really hoping that modu=
les are the final catalyst for the community to realize that CMake is this =
generations' Autotools. ;)</div><div><br></div><div>That said, I do thi=
nk it's worthwhile that folks at least understand where Abyx is coming =
from.<br></div><div><br></div><div>On Friday, February 26, 2016 at 11:57:44=
AM UTC-8, Richard Smith wrote:<br></div><blockquote class=3D"gmail_quote" =
style=3D"margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left=
:1ex">On Fri, Feb 26, 2016 at 7:23 AM, Abyx <<a rel=3D"nofollow">fl....@=
gmail.com</a>> wrote:
<br></blockquote><div><br></div><blockquote class=3D"gmail_quote" style=3D"=
margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,20=
4,204);border-left-style:solid;padding-left:1ex">1) You manually list the d=
ependencies between your libraries in your=C2=A0</blockquote><blockquote cl=
ass=3D"gmail_quote" style=3D"margin:0px 0px 0px 0.8ex;border-left-width:1px=
;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1e=
x">build rules. This is often a good thing for your code health, but may=C2=
=A0</blockquote><blockquote class=3D"gmail_quote" style=3D"margin:0px 0px 0=
px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-le=
ft-style:solid;padding-left:1ex">not be right for everyone.=C2=A0=C2=A0</bl=
ockquote><div><br></div><div>We're not talking about libraries! We'=
re talking about translation units and modules.</div><div><br></div><div>A =
single library could be made up of dozens and dozens (if not hundreds) of T=
Us and modules.</div><div>=C2=A0</div><blockquote class=3D"gmail_quote" sty=
le=3D"margin:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1e=
x">2) Your build process scans for import statements. This is not really
<br>any harder than scanning for #includes; no real parsing is required,
<br>at most, you need tokenize and maybe preprocess (if you want 'impor=
t
<br>MACRO_NAME' to work). The same is true for #includes. =C2=A0</block=
quote><div><br></div><div>This is much harder than scanning for #include fi=
les. No good modern C++ build chain actually has a separate "scan for =
includes" step anymore. :)<br></div></div></blockquote><div>Do you mea=
n scan by searching the text? Otherwise, the compiler may do some "sca=
n" work (as you said below).<br>It certainly should not be the right w=
ay. However, there are tools do=20
that in practice, e.g. Code::Blocks. (yep, not so modern & good...) <br=
>=C2=A0<br></div><blockquote class=3D"gmail_quote" style=3D"margin: 0;margi=
n-left: 0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;"><div dir=3D"l=
tr"><div></div><div><br></div><div>With headers, the dependency collection =
is just part of building. MSC, GCC, etc. all have flags that allow them to =
generate dependency chains while compiling, e.g. -MM -MF <file> on GC=
C. The order of compilation of TUs is usually irrelevant; in fact, it's=
safe to build a dependent without even knowing the dependency exists, so l=
ong as all the dependee files already exist. The dependencies are used only=
for checking whether files are out of date relative to another file.</div>=
<div><br></div><div>In fact, A.cpp and B.cpp can be compiled in any order, =
or compiled in parallel, or compiled on two different machines, even if B.c=
pp has a dependency on A.h.</div><div><br></div></div></blockquote><div>Agr=
eed. And g++ -MMD as a single step works as well, though the result=20
(makefile deps) may need additional parsing process, but not hard. I've=
actually=20
implemented it <a href=3D"https://bitbucket.org/FrankHB/yslib/src/2824dc90d=
869be7246b14abb6332e1997dd34a63/YFramework/source/NPL/Dependency.cpp?at=3Dm=
aster&fileviewer=3Dfile-view-default#Dependency.cpp-39">in several line=
s</a> with aid of my parsing library utilities and <a href=3D"https://bitbu=
cket.org/FrankHB/yslib/src/2824dc90d869be7246b14abb6332e1997dd34a63/Tools/S=
HBuild/Main.cpp?at=3Dmaster&fileviewer=3Dfile-view-default#Main.cpp-463=
">integrated it into my own implemented-in-one-file build tool</a>.<br>=C2=
=A0</div><blockquote class=3D"gmail_quote" style=3D"margin: 0;margin-left: =
0.8ex;border-left: 1px #ccc solid;padding-left: 1ex;"><div dir=3D"ltr"><div=
></div><div>With modules, this is no longer the case. The dependency graph =
must be present before any compilation begins, as building properly is now =
dependent on per-TU build artifacts (the .ifc files). Distributed builds re=
quire synchronization of the artifacts or island solving to ensure that fil=
es with dependency chains are all built on the same build node. This is bec=
ause B.cpp no longer would depend on A.h, but instead would depend on A.ifc=
, which doesn't even exist until after A.cpp is compiled.<br></div><div=
><br></div><div>Note in particular that this also means that with modules w=
e've just serialized a previously parallel process, which is theoretica=
lly a build deoptimization - seemingly the exact opposite of what many of u=
s are expecting or want out of modules!</div><div><br></div><div>That said,=
your option 3 is the obvious solution. It's roughly what most other mo=
dern languages with module systems do, and C++ implementations and the buil=
d tools we use are going to have to modernize to adopt to a world where the=
compiler plays a more integral role in the build chain than it did before.=
</div><div>=C2=A0<br></div><blockquote class=3D"gmail_quote" style=3D"margi=
n:0;margin-left:0.8ex;border-left:1px #ccc solid;padding-left:1ex">
<br>3) You ask your compiler to implicitly build (and cache) module
<br>interfaces on demand. This requires that your compiler has some way to
<br>map from an imported module name to the relevant interface file(s).
<br>That could happen via some implementation-defined means (such as
<br>Clang's module map files) or by making the module names directly
<br>correspond to module interface files (as suggested in
<br><a href=3D"http://www.google.com/url?q=3Dhttp%3A%2F%2Fwww.open-std.org%=
2Fjtc1%2Fsc22%2Fwg21%2Fdocs%2Fpapers%2F2016%2Fp0273r0.pdf&sa=3DD&sn=
tz=3D1&usg=3DAFQjCNFuSHtll19rhWabB9b_WnoR8Q0W_w" rel=3D"nofollow" targe=
t=3D"_blank" onmousedown=3D"this.href=3D'http://www.google.com/url?q\75=
http%3A%2F%2Fwww.open-std.org%2Fjtc1%2Fsc22%2Fwg21%2Fdocs%2Fpapers%2F2016%2=
Fp0273r0.pdf\46sa\75D\46sntz\0751\46usg\75AFQjCNFuSHtll19rhWabB9b_WnoR8Q0W_=
w';return true;" onclick=3D"this.href=3D'http://www.google.com/url?=
q\75http%3A%2F%2Fwww.open-std.org%2Fjtc1%2Fsc22%2Fwg21%2Fdocs%2Fpapers%2F20=
16%2Fp0273r0.pdf\46sa\75D\46sntz\0751\46usg\75AFQjCNFuSHtll19rhWabB9b_WnoR8=
Q0W_w';return true;">http://www.open-std.org/jtc1/<wbr>sc22/wg21/docs/p=
apers/2016/<wbr>p0273r0.pdf</a>).=C2=A0</blockquote></div></blockquote><div=
><br>I'm worrying there are many details not clear to average C++ users=
.. And comparing to the complexity of the specification, the whole system st=
ill seems to be too weak in functionality. Are there anything like <i>struc=
tures</i>, <i>signatures</i> and <i>functors</i> in module systems of ML-li=
ke languages proposed?<br><br>=C2=A0<br></div></div>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;ISO C++ Standard - Future Proposals" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:std-proposals+unsubscribe@isocpp.org">std-proposa=
ls+unsubscribe@isocpp.org</a>.<br />
To post to this group, send email to <a href=3D"mailto:std-proposals@isocpp=
..org">std-proposals@isocpp.org</a>.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/a/isocpp.org/d/msgid/std-proposals/29edd7b0-671f-49ea-8538-46070981228f%=
40isocpp.org?utm_medium=3Demail&utm_source=3Dfooter">https://groups.google.=
com/a/isocpp.org/d/msgid/std-proposals/29edd7b0-671f-49ea-8538-46070981228f=
%40isocpp.org</a>.<br />
------=_Part_793_1406381613.1456728161680--
------=_Part_792_197808656.1456728161680--
.
Author: Richard Smith <richard@metafoo.co.uk>
Date: Mon, 29 Feb 2016 05:39:00 -0800
Raw View
On Sun, Feb 28, 2016 at 5:41 PM, Sean Middleditch
<sean.middleditch@gmail.com> wrote:
> I'll start off by saying that I don't think that Abyx's worries are a
> significant problem. Yes, build systems will need to change, but they've
> needed to change for 20 years now. If modules are the impetus for the
> community to finally make building C++ not ridiculous, that's just fine with
> me. I'm actually really hoping that modules are the final catalyst for the
> community to realize that CMake is this generations' Autotools. ;)
>
> That said, I do think it's worthwhile that folks at least understand where
> Abyx is coming from.
>
> On Friday, February 26, 2016 at 11:57:44 AM UTC-8, Richard Smith wrote:
>>
>> On Fri, Feb 26, 2016 at 7:23 AM, Abyx <fl....@gmail.com> wrote:
>
>
>> 1) You manually list the dependencies between your libraries in your
>>
>> build rules. This is often a good thing for your code health, but may
>>
>> not be right for everyone.
>
>
> We're not talking about libraries! We're talking about translation units and
> modules.
We're talking about exactly the things that you configure in your
build tool as a library -- that is, a small number of header files
plus their corresponding source files. (These are not necessarily the
same size as .so / .dll files / executables, which would typically be
composed of a number of these build-system-level libraries.)
> A single library could be made up of dozens and dozens (if not hundreds) of
> TUs and modules.
>
>>
>> 2) Your build process scans for import statements. This is not really
>> any harder than scanning for #includes; no real parsing is required,
>> at most, you need tokenize and maybe preprocess (if you want 'import
>> MACRO_NAME' to work). The same is true for #includes.
>
>
> This is much harder than scanning for #include files. No good modern C++
> build chain actually has a separate "scan for includes" step anymore. :)
The only way that's true is if you define "good modern" to make it
true. Scons, Jam, Bazel, ... all support include scanning.
> With headers, the dependency collection is just part of building. MSC, GCC,
> etc. all have flags that allow them to generate dependency chains while
> compiling, e.g. -MM -MF <file> on GCC. The order of compilation of TUs is
> usually irrelevant; in fact, it's safe to build a dependent without even
> knowing the dependency exists, so long as all the dependee files already
> exist. The dependencies are used only for checking whether files are out of
> date relative to another file.
That approach does not work for large-scale distributed builds where
the set of files that are inputs to the build must be known before the
build action begins (so they can be staged to the build farm).
> Note in particular that this also means that with modules we've just
> serialized a previously parallel process, which is theoretically a build
> deoptimization - seemingly the exact opposite of what many of us are
> expecting or want out of modules!
Yes, that's theoretically true, but does not match our experience in
practice when building large existing codebases with (Clang's
implementation of) modules. While use of modules does cause there to
be a longer dependency chain for builds (compilation of Z.cpp depends
on Y1.pcm, Y2.pcm, ... already having been compiled, which depend on
X1.pcm, X2.pcm, ... already having been compiled, ...), and the total
amount of work required is somewhat higher than compiling Z.cpp
directly due to the serialization / deserialization overhead, those
individual steps are parallelizable, the work to build them is reused,
and despite the longer dependency chain we see significant build time
reductions even for clean builds.
> That said, your option 3 is the obvious solution. It's roughly what most
> other modern languages with module systems do, and C++ implementations and
> the build tools we use are going to have to modernize to adopt to a world
> where the compiler plays a more integral role in the build chain than it did
> before.
The downside is that option 3 parallelizes extremely poorly. If you
want module builds to be parallelized, distributed, and each only
performed once, you should look at a different option. If you only
care about local builds, we've found that it can be effective.
>> 3) You ask your compiler to implicitly build (and cache) module
>> interfaces on demand. This requires that your compiler has some way to
>> map from an imported module name to the relevant interface file(s).
>> That could happen via some implementation-defined means (such as
>> Clang's module map files) or by making the module names directly
>> correspond to module interface files (as suggested in
>> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0273r0.pdf).
>
> --
> You received this message because you are subscribed to the Google Groups
> "ISO C++ Standard - Future Proposals" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to std-proposals+unsubscribe@isocpp.org.
> To post to this group, send email to std-proposals@isocpp.org.
> To view this discussion on the web visit
> https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/d30abb01-3e49-424a-a410-556b60b1b833%40isocpp.org.
--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/CAOfiQqnL7tw3y6v5D-hBfbxQhE-w8EXDC-Nr%2BYjXowVg97DJnA%40mail.gmail.com.
.
Author: Abyx <fl.lllj@gmail.com>
Date: Mon, 29 Feb 2016 07:30:37 -0800 (PST)
Raw View
------=_Part_3590_281349401.1456759837621
Content-Type: multipart/alternative;
boundary="----=_Part_3591_1413094598.1456759837625"
------=_Part_3591_1413094598.1456759837625
Content-Type: text/plain; charset=UTF-8
Let's clarify how we're going to use modules. There is an opinion that
modules will completely replace header files.
Many code bases have header per class, e.g. widget.h, widget.cpp, button.h,
button.cpp, label.h, label.cpp
And because module exports can be only in a single file, that means we are
going to have (sub)module per class, e.g. modules widget, label, button,
etc.
Not let's look at some decent mature code base, e.g. chromium -
https://code.google.com/p/chromium/codesearch#chromium/src/net/http/http_stream_factory_impl_job.cc
This file has around 50 fine-grained include directives. With modules it
will be 50 imports.
Some files have more than hundred includes, most of the files have more
than ten includes.
No one in their right mind will copy all those dependencies into a build
script.
That rules out (1).
2) Build tool cannot scan for includes by itself. For imports it needs full
preprocessing (because of #ifdef and import statements in included files).
In order to do so it needs all the defines, include paths and implicit
(forced) includes from compiler-specific environment variables and global
(system) configuration files.
The ninja build tool has a nice explanation of this process -
https://ninja-build.org/manual.html#ref_headers
There is one more issue - even if compiler would have a --list-imports
switch, the build tool would have process files twice.
First time - to get the dependencies, second time - to compile it after the
dependencies were built.
cc --list-imports a.cpp
cc --list-imports dependency1.ixx
...
cc --list-imports dependencyN.ixx
cc dependencyN.ixx
....
cc dependency1.ixx
cc a.cpp
And (3) would just embed a build tool into a compiler - we take a root .cpp
file which imports all the other modules, pass it to a compiler, and
compiler builds all the other TU.
That would be great except that now it's impossible to provide custom
compiler options to certain TU.
Sorry for being so negative.
On Monday, February 29, 2016 at 4:39:03 PM UTC+3, Richard Smith wrote:
>
> On Sun, Feb 28, 2016 at 5:41 PM, Sean Middleditch
> <sean.mid...@gmail.com <javascript:>> wrote:
> > I'll start off by saying that I don't think that Abyx's worries are a
> > significant problem. Yes, build systems will need to change, but they've
> > needed to change for 20 years now. If modules are the impetus for the
> > community to finally make building C++ not ridiculous, that's just fine
> with
> > me. I'm actually really hoping that modules are the final catalyst for
> the
> > community to realize that CMake is this generations' Autotools. ;)
> >
> > That said, I do think it's worthwhile that folks at least understand
> where
> > Abyx is coming from.
> >
> > On Friday, February 26, 2016 at 11:57:44 AM UTC-8, Richard Smith wrote:
> >>
> >> On Fri, Feb 26, 2016 at 7:23 AM, Abyx <fl....@gmail.com> wrote:
> >
> >
> >> 1) You manually list the dependencies between your libraries in your
> >>
> >> build rules. This is often a good thing for your code health, but may
> >>
> >> not be right for everyone.
> >
> >
> > We're not talking about libraries! We're talking about translation units
> and
> > modules.
>
> We're talking about exactly the things that you configure in your
> build tool as a library -- that is, a small number of header files
> plus their corresponding source files. (These are not necessarily the
> same size as .so / .dll files / executables, which would typically be
> composed of a number of these build-system-level libraries.)
>
> > A single library could be made up of dozens and dozens (if not hundreds)
> of
> > TUs and modules.
> >
> >>
> >> 2) Your build process scans for import statements. This is not really
> >> any harder than scanning for #includes; no real parsing is required,
> >> at most, you need tokenize and maybe preprocess (if you want 'import
> >> MACRO_NAME' to work). The same is true for #includes.
> >
> >
> > This is much harder than scanning for #include files. No good modern C++
> > build chain actually has a separate "scan for includes" step anymore. :)
>
> The only way that's true is if you define "good modern" to make it
> true. Scons, Jam, Bazel, ... all support include scanning.
>
> > With headers, the dependency collection is just part of building. MSC,
> GCC,
> > etc. all have flags that allow them to generate dependency chains while
> > compiling, e.g. -MM -MF <file> on GCC. The order of compilation of TUs
> is
> > usually irrelevant; in fact, it's safe to build a dependent without even
> > knowing the dependency exists, so long as all the dependee files already
> > exist. The dependencies are used only for checking whether files are out
> of
> > date relative to another file.
>
> That approach does not work for large-scale distributed builds where
> the set of files that are inputs to the build must be known before the
> build action begins (so they can be staged to the build farm).
>
> > Note in particular that this also means that with modules we've just
> > serialized a previously parallel process, which is theoretically a build
> > deoptimization - seemingly the exact opposite of what many of us are
> > expecting or want out of modules!
>
> Yes, that's theoretically true, but does not match our experience in
> practice when building large existing codebases with (Clang's
> implementation of) modules. While use of modules does cause there to
> be a longer dependency chain for builds (compilation of Z.cpp depends
> on Y1.pcm, Y2.pcm, ... already having been compiled, which depend on
> X1.pcm, X2.pcm, ... already having been compiled, ...), and the total
> amount of work required is somewhat higher than compiling Z.cpp
> directly due to the serialization / deserialization overhead, those
> individual steps are parallelizable, the work to build them is reused,
> and despite the longer dependency chain we see significant build time
> reductions even for clean builds.
>
> > That said, your option 3 is the obvious solution. It's roughly what most
> > other modern languages with module systems do, and C++ implementations
> and
> > the build tools we use are going to have to modernize to adopt to a
> world
> > where the compiler plays a more integral role in the build chain than it
> did
> > before.
>
> The downside is that option 3 parallelizes extremely poorly. If you
> want module builds to be parallelized, distributed, and each only
> performed once, you should look at a different option. If you only
> care about local builds, we've found that it can be effective.
>
> >> 3) You ask your compiler to implicitly build (and cache) module
> >> interfaces on demand. This requires that your compiler has some way to
> >> map from an imported module name to the relevant interface file(s).
> >> That could happen via some implementation-defined means (such as
> >> Clang's module map files) or by making the module names directly
> >> correspond to module interface files (as suggested in
> >> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0273r0.pdf).
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups
> > "ISO C++ Standard - Future Proposals" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an
> > email to std-proposal...@isocpp.org <javascript:>.
> > To post to this group, send email to std-pr...@isocpp.org <javascript:>.
>
> > To view this discussion on the web visit
> >
> https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/d30abb01-3e49-424a-a410-556b60b1b833%40isocpp.org.
>
>
--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/d60dca15-396e-4886-92bf-3c9b79bbb829%40isocpp.org.
------=_Part_3591_1413094598.1456759837625
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">Let's clarify how we're going to use modules. Ther=
e is an opinion that modules will completely replace header files.<br>Many =
code bases have header per class, e.g. widget.h, widget.cpp, button.h, butt=
on.cpp, label.h, label.cpp<br>And because module exports can be only in a s=
ingle file, that means we are going to have (sub)module per class, e.g. mod=
ules widget, label, button, etc.<br><br>Not let's look at some decent m=
ature code base, e.g. chromium - https://code.google.com/p/chromium/codesea=
rch#chromium/src/net/http/http_stream_factory_impl_job.cc<br>This file has =
around 50 fine-grained include directives. With modules it will be 50 impor=
ts.<br>Some files have more than hundred includes, most of the files have m=
ore than ten includes.<br><br>No one in their right mind will copy all thos=
e dependencies into a build script.<br>That rules out (1).<br><br>2) Build =
tool cannot scan for includes by itself. For imports it needs full preproce=
ssing (because of #ifdef and import statements in included files).<br>In or=
der to do so it needs all the defines, include paths and implicit (forced) =
includes from compiler-specific environment variables and global (system) c=
onfiguration files.<br>The ninja build tool has a nice explanation of this =
process - https://ninja-build.org/manual.html#ref_headers<br><br>There is o=
ne more issue - even if compiler would have a --list-imports switch, the bu=
ild tool would have process files twice.<br>First time - to get the depende=
ncies, second time - to compile it after the dependencies were built.<br><b=
r>cc --list-imports a.cpp<br>cc --list-imports dependency1.ixx<br>..<br>cc =
--list-imports dependencyN.ixx<br>cc dependencyN.ixx<br>...<br>cc dependenc=
y1.ixx<br>cc a.cpp<br><br>And (3) would just embed a build tool into a comp=
iler - we take a root .cpp file which imports all the other modules, pass i=
t to a compiler, and compiler builds all the other TU.<br>That would be gre=
at except that now it's impossible to provide custom compiler options t=
o certain TU.<br><br>Sorry for being so negative.<br><br><br>On Monday, Feb=
ruary 29, 2016 at 4:39:03 PM UTC+3, Richard Smith wrote:<blockquote class=
=3D"gmail_quote" style=3D"margin: 0;margin-left: 0.8ex;border-left: 1px #cc=
c solid;padding-left: 1ex;">On Sun, Feb 28, 2016 at 5:41 PM, Sean Middledit=
ch
<br><<a href=3D"javascript:" target=3D"_blank" gdf-obfuscated-mailto=3D"=
rmB2AHDSAwAJ" rel=3D"nofollow" onmousedown=3D"this.href=3D'javascript:&=
#39;;return true;" onclick=3D"this.href=3D'javascript:';return true=
;">sean.mid...@gmail.com</a>> wrote:
<br>> I'll start off by saying that I don't think that Abyx'=
s worries are a
<br>> significant problem. Yes, build systems will need to change, but t=
hey've
<br>> needed to change for 20 years now. If modules are the impetus for =
the
<br>> community to finally make building C++ not ridiculous, that's =
just fine with
<br>> me. I'm actually really hoping that modules are the final cata=
lyst for the
<br>> community to realize that CMake is this generations' Autotools=
.. ;)
<br>>
<br>> That said, I do think it's worthwhile that folks at least unde=
rstand where
<br>> Abyx is coming from.
<br>>
<br>> On Friday, February 26, 2016 at 11:57:44 AM UTC-8, Richard Smith w=
rote:
<br>>>
<br>>> On Fri, Feb 26, 2016 at 7:23 AM, Abyx <<a>fl....@gmail.com<=
/a>> wrote:
<br>>
<br>>
<br>>> 1) You manually list the dependencies between your libraries i=
n your
<br>>>
<br>>> build rules. This is often a good thing for your code health, =
but may
<br>>>
<br>>> not be right for everyone.
<br>>
<br>>
<br>> We're not talking about libraries! We're talking about tra=
nslation units and
<br>> modules.
<br>
<br>We're talking about exactly the things that you configure in your
<br>build tool as a library -- that is, a small number of header files
<br>plus their corresponding source files. (These are not necessarily the
<br>same size as .so / .dll files / executables, which would typically be
<br>composed of a number of these build-system-level libraries.)
<br>
<br>> A single library could be made up of dozens and dozens (if not hun=
dreds) of
<br>> TUs and modules.
<br>>
<br>>>
<br>>> 2) Your build process scans for import statements. This is not=
really
<br>>> any harder than scanning for #includes; no real parsing is req=
uired,
<br>>> at most, you need tokenize and maybe preprocess (if you want &=
#39;import
<br>>> MACRO_NAME' to work). The same is true for #includes.
<br>>
<br>>
<br>> This is much harder than scanning for #include files. No good mode=
rn C++
<br>> build chain actually has a separate "scan for includes" =
step anymore. :)
<br>
<br>The only way that's true is if you define "good modern" t=
o make it
<br>true. Scons, Jam, Bazel, ... all support include scanning.
<br>
<br>> With headers, the dependency collection is just part of building. =
MSC, GCC,
<br>> etc. all have flags that allow them to generate dependency chains =
while
<br>> compiling, e.g. -MM -MF <file> on GCC. The order of compilat=
ion of TUs is
<br>> usually irrelevant; in fact, it's safe to build a dependent wi=
thout even
<br>> knowing the dependency exists, so long as all the dependee files a=
lready
<br>> exist. The dependencies are used only for checking whether files a=
re out of
<br>> date relative to another file.
<br>
<br>That approach does not work for large-scale distributed builds where
<br>the set of files that are inputs to the build must be known before the
<br>build action begins (so they can be staged to the build farm).
<br>
<br>> Note in particular that this also means that with modules we'v=
e just
<br>> serialized a previously parallel process, which is theoretically a=
build
<br>> deoptimization - seemingly the exact opposite of what many of us a=
re
<br>> expecting or want out of modules!
<br>
<br>Yes, that's theoretically true, but does not match our experience i=
n
<br>practice when building large existing codebases with (Clang's
<br>implementation of) modules. While use of modules does cause there to
<br>be a longer dependency chain for builds (compilation of Z.cpp depends
<br>on Y1.pcm, Y2.pcm, ... already having been compiled, which depend on
<br>X1.pcm, X2.pcm, ... already having been compiled, ...), and the total
<br>amount of work required is somewhat higher than compiling Z.cpp
<br>directly due to the serialization / deserialization overhead, those
<br>individual steps are parallelizable, the work to build them is reused,
<br>and despite the longer dependency chain we see significant build time
<br>reductions even for clean builds.
<br>
<br>> That said, your option 3 is the obvious solution. It's roughly=
what most
<br>> other modern languages with module systems do, and C++ implementat=
ions and
<br>> the build tools we use are going to have to modernize to adopt to =
a world
<br>> where the compiler plays a more integral role in the build chain t=
han it did
<br>> before.
<br>
<br>The downside is that option 3 parallelizes extremely poorly. If you
<br>want module builds to be parallelized, distributed, and each only
<br>performed once, you should look at a different option. If you only
<br>care about local builds, we've found that it can be effective.
<br>
<br>>> 3) You ask your compiler to implicitly build (and cache) modul=
e
<br>>> interfaces on demand. This requires that your compiler has som=
e way to
<br>>> map from an imported module name to the relevant interface fil=
e(s).
<br>>> That could happen via some implementation-defined means (such =
as
<br>>> Clang's module map files) or by making the module names di=
rectly
<br>>> correspond to module interface files (as suggested in
<br>>> <a href=3D"http://www.open-std.org/jtc1/sc22/wg21/docs/papers/=
2016/p0273r0.pdf" target=3D"_blank" rel=3D"nofollow" onmousedown=3D"this.hr=
ef=3D'http://www.google.com/url?q\75http%3A%2F%2Fwww.open-std.org%2Fjtc=
1%2Fsc22%2Fwg21%2Fdocs%2Fpapers%2F2016%2Fp0273r0.pdf\46sa\75D\46sntz\0751\4=
6usg\75AFQjCNFuSHtll19rhWabB9b_WnoR8Q0W_w';return true;" onclick=3D"thi=
s.href=3D'http://www.google.com/url?q\75http%3A%2F%2Fwww.open-std.org%2=
Fjtc1%2Fsc22%2Fwg21%2Fdocs%2Fpapers%2F2016%2Fp0273r0.pdf\46sa\75D\46sntz\07=
51\46usg\75AFQjCNFuSHtll19rhWabB9b_WnoR8Q0W_w';return true;">http://www=
..open-std.org/jtc1/<wbr>sc22/wg21/docs/papers/2016/<wbr>p0273r0.pdf</a>).
<br>>
<br>> --
<br>> You received this message because you are subscribed to the Google=
Groups
<br>> "ISO C++ Standard - Future Proposals" group.
<br>> To unsubscribe from this group and stop receiving emails from it, =
send an
<br>> email to <a href=3D"javascript:" target=3D"_blank" gdf-obfuscated-=
mailto=3D"rmB2AHDSAwAJ" rel=3D"nofollow" onmousedown=3D"this.href=3D'ja=
vascript:';return true;" onclick=3D"this.href=3D'javascript:';r=
eturn true;">std-proposal...@<wbr>isocpp.org</a>.
<br>> To post to this group, send email to <a href=3D"javascript:" targe=
t=3D"_blank" gdf-obfuscated-mailto=3D"rmB2AHDSAwAJ" rel=3D"nofollow" onmous=
edown=3D"this.href=3D'javascript:';return true;" onclick=3D"this.hr=
ef=3D'javascript:';return true;">std-pr...@isocpp.org</a>.
<br>> To view this discussion on the web visit
<br>> <a href=3D"https://groups.google.com/a/isocpp.org/d/msgid/std-prop=
osals/d30abb01-3e49-424a-a410-556b60b1b833%40isocpp.org" target=3D"_blank" =
rel=3D"nofollow" onmousedown=3D"this.href=3D'https://groups.google.com/=
a/isocpp.org/d/msgid/std-proposals/d30abb01-3e49-424a-a410-556b60b1b833%40i=
socpp.org';return true;" onclick=3D"this.href=3D'https://groups.goo=
gle.com/a/isocpp.org/d/msgid/std-proposals/d30abb01-3e49-424a-a410-556b60b1=
b833%40isocpp.org';return true;">https://groups.google.com/a/<wbr>isocp=
p.org/d/msgid/std-<wbr>proposals/d30abb01-3e49-424a-<wbr>a410-556b60b1b833%=
40isocpp.org</a><wbr>.
<br></blockquote></div>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;ISO C++ Standard - Future Proposals" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:std-proposals+unsubscribe@isocpp.org">std-proposa=
ls+unsubscribe@isocpp.org</a>.<br />
To post to this group, send email to <a href=3D"mailto:std-proposals@isocpp=
..org">std-proposals@isocpp.org</a>.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/a/isocpp.org/d/msgid/std-proposals/d60dca15-396e-4886-92bf-3c9b79bbb829%=
40isocpp.org?utm_medium=3Demail&utm_source=3Dfooter">https://groups.google.=
com/a/isocpp.org/d/msgid/std-proposals/d60dca15-396e-4886-92bf-3c9b79bbb829=
%40isocpp.org</a>.<br />
------=_Part_3591_1413094598.1456759837625--
------=_Part_3590_281349401.1456759837621--
.
Author: Abyx <fl.lllj@gmail.com>
Date: Mon, 29 Feb 2016 07:32:16 -0800 (PST)
Raw View
------=_Part_149_65210392.1456759936521
Content-Type: multipart/alternative;
boundary="----=_Part_150_135361035.1456759936521"
------=_Part_150_135361035.1456759936521
Content-Type: text/plain; charset=UTF-8
Let's clarify how we're going to use modules. There is an opinion that
modules will completely replace header files.
Many code bases have header per class, e.g. widget.h, widget.cpp, button.h,
button.cpp, label.h, label.cpp
And because module exports can be only in a single file, that means we are
going to have (sub)module per class, e.g. modules widget, label, button,
etc.
Not let's look at some decent mature code base, e.g. chromium -
https://code.google.com/p/chromium/codesearch#chromium/src/net/http/http_stream_factory_impl_job.cc
This file has around 50 fine-grained include directives. With modules it
will be 50 imports.
Some files have more than hundred includes, most of the files have more
than ten includes.
No one in their right mind will copy all those dependencies into a build
script.
That rules out (1).
2) Build tool cannot scan for includes by itself. For imports it needs full
preprocessing (because of #ifdef and import statements in included files).
In order to do so it needs all the defines, include paths and implicit
(forced) includes from compiler-specific environment variables and global
(system) configuration files.
The ninja build tool has a nice explanation of this process -
https://ninja-build.org/manual.html#ref_headers
There is one more issue - even if compiler would have a --list-imports
switch, the build tool would have process files twice.
First time - to get the dependencies, second time - to compile it after the
dependencies were built.
cc --list-imports a.cpp
cc --list-imports dependency1.ixx
...
cc --list-imports dependencyN.ixx
cc dependencyN.ixx
....
cc dependency1.ixx
cc a.cpp
And (3) would just embed a build tool into a compiler - we take a root .cpp
file which imports all the other modules, pass it to a compiler, and
compiler builds all the other TU.
That would be great except that now it's impossible to provide custom
compiler options to certain TU.
Sorry for being so negative.
On Monday, February 29, 2016 at 4:39:03 PM UTC+3, Richard Smith wrote:
>
> On Sun, Feb 28, 2016 at 5:41 PM, Sean Middleditch
> <sean.mid...@gmail.com <javascript:>> wrote:
> > I'll start off by saying that I don't think that Abyx's worries are a
> > significant problem. Yes, build systems will need to change, but they've
> > needed to change for 20 years now. If modules are the impetus for the
> > community to finally make building C++ not ridiculous, that's just fine
> with
> > me. I'm actually really hoping that modules are the final catalyst for
> the
> > community to realize that CMake is this generations' Autotools. ;)
> >
> > That said, I do think it's worthwhile that folks at least understand
> where
> > Abyx is coming from.
> >
> > On Friday, February 26, 2016 at 11:57:44 AM UTC-8, Richard Smith wrote:
> >>
> >> On Fri, Feb 26, 2016 at 7:23 AM, Abyx <fl....@gmail.com> wrote:
> >
> >
> >> 1) You manually list the dependencies between your libraries in your
> >>
> >> build rules. This is often a good thing for your code health, but may
> >>
> >> not be right for everyone.
> >
> >
> > We're not talking about libraries! We're talking about translation units
> and
> > modules.
>
> We're talking about exactly the things that you configure in your
> build tool as a library -- that is, a small number of header files
> plus their corresponding source files. (These are not necessarily the
> same size as .so / .dll files / executables, which would typically be
> composed of a number of these build-system-level libraries.)
>
> > A single library could be made up of dozens and dozens (if not hundreds)
> of
> > TUs and modules.
> >
> >>
> >> 2) Your build process scans for import statements. This is not really
> >> any harder than scanning for #includes; no real parsing is required,
> >> at most, you need tokenize and maybe preprocess (if you want 'import
> >> MACRO_NAME' to work). The same is true for #includes.
> >
> >
> > This is much harder than scanning for #include files. No good modern C++
> > build chain actually has a separate "scan for includes" step anymore. :)
>
> The only way that's true is if you define "good modern" to make it
> true. Scons, Jam, Bazel, ... all support include scanning.
>
> > With headers, the dependency collection is just part of building. MSC,
> GCC,
> > etc. all have flags that allow them to generate dependency chains while
> > compiling, e.g. -MM -MF <file> on GCC. The order of compilation of TUs
> is
> > usually irrelevant; in fact, it's safe to build a dependent without even
> > knowing the dependency exists, so long as all the dependee files already
> > exist. The dependencies are used only for checking whether files are out
> of
> > date relative to another file.
>
> That approach does not work for large-scale distributed builds where
> the set of files that are inputs to the build must be known before the
> build action begins (so they can be staged to the build farm).
>
> > Note in particular that this also means that with modules we've just
> > serialized a previously parallel process, which is theoretically a build
> > deoptimization - seemingly the exact opposite of what many of us are
> > expecting or want out of modules!
>
> Yes, that's theoretically true, but does not match our experience in
> practice when building large existing codebases with (Clang's
> implementation of) modules. While use of modules does cause there to
> be a longer dependency chain for builds (compilation of Z.cpp depends
> on Y1.pcm, Y2.pcm, ... already having been compiled, which depend on
> X1.pcm, X2.pcm, ... already having been compiled, ...), and the total
> amount of work required is somewhat higher than compiling Z.cpp
> directly due to the serialization / deserialization overhead, those
> individual steps are parallelizable, the work to build them is reused,
> and despite the longer dependency chain we see significant build time
> reductions even for clean builds.
>
> > That said, your option 3 is the obvious solution. It's roughly what most
> > other modern languages with module systems do, and C++ implementations
> and
> > the build tools we use are going to have to modernize to adopt to a
> world
> > where the compiler plays a more integral role in the build chain than it
> did
> > before.
>
> The downside is that option 3 parallelizes extremely poorly. If you
> want module builds to be parallelized, distributed, and each only
> performed once, you should look at a different option. If you only
> care about local builds, we've found that it can be effective.
>
> >> 3) You ask your compiler to implicitly build (and cache) module
> >> interfaces on demand. This requires that your compiler has some way to
> >> map from an imported module name to the relevant interface file(s).
> >> That could happen via some implementation-defined means (such as
> >> Clang's module map files) or by making the module names directly
> >> correspond to module interface files (as suggested in
> >> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0273r0.pdf).
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups
> > "ISO C++ Standard - Future Proposals" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an
> > email to std-proposal...@isocpp.org <javascript:>.
> > To post to this group, send email to std-pr...@isocpp.org <javascript:>.
>
> > To view this discussion on the web visit
> >
> https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/d30abb01-3e49-424a-a410-556b60b1b833%40isocpp.org.
>
>
--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/bc32cd95-7011-479c-85eb-a4ebe1777cd8%40isocpp.org.
------=_Part_150_135361035.1456759936521
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
<div dir=3D"ltr">Let's clarify how we're going to use modules. Ther=
e is an opinion that modules will completely replace header files.<br>Many =
code bases have header per class, e.g. widget.h, widget.cpp, button.h, butt=
on.cpp, label.h, label.cpp<br>And because module exports can be only in a s=
ingle file, that means we are going to have (sub)module per class, e.g. mod=
ules widget, label, button, etc.<br><br>Not let's look at some decent m=
ature code base, e.g. chromium - https://code.google.com/p/chromium/codesea=
rch#chromium/src/net/http/http_stream_factory_impl_job.cc<br>This file has =
around 50 fine-grained include directives. With modules it will be 50 impor=
ts.<br>Some files have more than hundred includes, most of the files have m=
ore than ten includes.<br><br>No one in their right mind will copy all thos=
e dependencies into a build script.<br>That rules out (1).<br><br>2) Build =
tool cannot scan for includes by itself. For imports it needs full preproce=
ssing (because of #ifdef and import statements in included files).<br>In or=
der to do so it needs all the defines, include paths and implicit (forced) =
includes from compiler-specific environment variables and global (system) c=
onfiguration files.<br>The ninja build tool has a nice explanation of this =
process - https://ninja-build.org/manual.html#ref_headers<br><br>There is o=
ne more issue - even if compiler would have a --list-imports switch, the bu=
ild tool would have process files twice.<br>First time - to get the depende=
ncies, second time - to compile it after the dependencies were built.<br><b=
r>cc --list-imports a.cpp<br>cc --list-imports dependency1.ixx<br>..<br>cc =
--list-imports dependencyN.ixx<br>cc dependencyN.ixx<br>...<br>cc dependenc=
y1.ixx<br>cc a.cpp<br><br>And (3) would just embed a build tool into a comp=
iler - we take a root .cpp file which imports all the other modules, pass i=
t to a compiler, and compiler builds all the other TU.<br>That would be gre=
at except that now it's impossible to provide custom compiler options t=
o certain TU.<br><br>Sorry for being so negative.<br><br><br>On Monday, Feb=
ruary 29, 2016 at 4:39:03 PM UTC+3, Richard Smith wrote:<blockquote class=
=3D"gmail_quote" style=3D"margin: 0;margin-left: 0.8ex;border-left: 1px #cc=
c solid;padding-left: 1ex;">On Sun, Feb 28, 2016 at 5:41 PM, Sean Middledit=
ch
<br><<a href=3D"javascript:" target=3D"_blank" gdf-obfuscated-mailto=3D"=
rmB2AHDSAwAJ" rel=3D"nofollow" onmousedown=3D"this.href=3D'javascript:&=
#39;;return true;" onclick=3D"this.href=3D'javascript:';return true=
;">sean.mid...@gmail.com</a>> wrote:
<br>> I'll start off by saying that I don't think that Abyx'=
s worries are a
<br>> significant problem. Yes, build systems will need to change, but t=
hey've
<br>> needed to change for 20 years now. If modules are the impetus for =
the
<br>> community to finally make building C++ not ridiculous, that's =
just fine with
<br>> me. I'm actually really hoping that modules are the final cata=
lyst for the
<br>> community to realize that CMake is this generations' Autotools=
.. ;)
<br>>
<br>> That said, I do think it's worthwhile that folks at least unde=
rstand where
<br>> Abyx is coming from.
<br>>
<br>> On Friday, February 26, 2016 at 11:57:44 AM UTC-8, Richard Smith w=
rote:
<br>>>
<br>>> On Fri, Feb 26, 2016 at 7:23 AM, Abyx <<a>fl....@gmail.com<=
/a>> wrote:
<br>>
<br>>
<br>>> 1) You manually list the dependencies between your libraries i=
n your
<br>>>
<br>>> build rules. This is often a good thing for your code health, =
but may
<br>>>
<br>>> not be right for everyone.
<br>>
<br>>
<br>> We're not talking about libraries! We're talking about tra=
nslation units and
<br>> modules.
<br>
<br>We're talking about exactly the things that you configure in your
<br>build tool as a library -- that is, a small number of header files
<br>plus their corresponding source files. (These are not necessarily the
<br>same size as .so / .dll files / executables, which would typically be
<br>composed of a number of these build-system-level libraries.)
<br>
<br>> A single library could be made up of dozens and dozens (if not hun=
dreds) of
<br>> TUs and modules.
<br>>
<br>>>
<br>>> 2) Your build process scans for import statements. This is not=
really
<br>>> any harder than scanning for #includes; no real parsing is req=
uired,
<br>>> at most, you need tokenize and maybe preprocess (if you want &=
#39;import
<br>>> MACRO_NAME' to work). The same is true for #includes.
<br>>
<br>>
<br>> This is much harder than scanning for #include files. No good mode=
rn C++
<br>> build chain actually has a separate "scan for includes" =
step anymore. :)
<br>
<br>The only way that's true is if you define "good modern" t=
o make it
<br>true. Scons, Jam, Bazel, ... all support include scanning.
<br>
<br>> With headers, the dependency collection is just part of building. =
MSC, GCC,
<br>> etc. all have flags that allow them to generate dependency chains =
while
<br>> compiling, e.g. -MM -MF <file> on GCC. The order of compilat=
ion of TUs is
<br>> usually irrelevant; in fact, it's safe to build a dependent wi=
thout even
<br>> knowing the dependency exists, so long as all the dependee files a=
lready
<br>> exist. The dependencies are used only for checking whether files a=
re out of
<br>> date relative to another file.
<br>
<br>That approach does not work for large-scale distributed builds where
<br>the set of files that are inputs to the build must be known before the
<br>build action begins (so they can be staged to the build farm).
<br>
<br>> Note in particular that this also means that with modules we'v=
e just
<br>> serialized a previously parallel process, which is theoretically a=
build
<br>> deoptimization - seemingly the exact opposite of what many of us a=
re
<br>> expecting or want out of modules!
<br>
<br>Yes, that's theoretically true, but does not match our experience i=
n
<br>practice when building large existing codebases with (Clang's
<br>implementation of) modules. While use of modules does cause there to
<br>be a longer dependency chain for builds (compilation of Z.cpp depends
<br>on Y1.pcm, Y2.pcm, ... already having been compiled, which depend on
<br>X1.pcm, X2.pcm, ... already having been compiled, ...), and the total
<br>amount of work required is somewhat higher than compiling Z.cpp
<br>directly due to the serialization / deserialization overhead, those
<br>individual steps are parallelizable, the work to build them is reused,
<br>and despite the longer dependency chain we see significant build time
<br>reductions even for clean builds.
<br>
<br>> That said, your option 3 is the obvious solution. It's roughly=
what most
<br>> other modern languages with module systems do, and C++ implementat=
ions and
<br>> the build tools we use are going to have to modernize to adopt to =
a world
<br>> where the compiler plays a more integral role in the build chain t=
han it did
<br>> before.
<br>
<br>The downside is that option 3 parallelizes extremely poorly. If you
<br>want module builds to be parallelized, distributed, and each only
<br>performed once, you should look at a different option. If you only
<br>care about local builds, we've found that it can be effective.
<br>
<br>>> 3) You ask your compiler to implicitly build (and cache) modul=
e
<br>>> interfaces on demand. This requires that your compiler has som=
e way to
<br>>> map from an imported module name to the relevant interface fil=
e(s).
<br>>> That could happen via some implementation-defined means (such =
as
<br>>> Clang's module map files) or by making the module names di=
rectly
<br>>> correspond to module interface files (as suggested in
<br>>> <a href=3D"http://www.open-std.org/jtc1/sc22/wg21/docs/papers/=
2016/p0273r0.pdf" target=3D"_blank" rel=3D"nofollow" onmousedown=3D"this.hr=
ef=3D'http://www.google.com/url?q\75http%3A%2F%2Fwww.open-std.org%2Fjtc=
1%2Fsc22%2Fwg21%2Fdocs%2Fpapers%2F2016%2Fp0273r0.pdf\46sa\75D\46sntz\0751\4=
6usg\75AFQjCNFuSHtll19rhWabB9b_WnoR8Q0W_w';return true;" onclick=3D"thi=
s.href=3D'http://www.google.com/url?q\75http%3A%2F%2Fwww.open-std.org%2=
Fjtc1%2Fsc22%2Fwg21%2Fdocs%2Fpapers%2F2016%2Fp0273r0.pdf\46sa\75D\46sntz\07=
51\46usg\75AFQjCNFuSHtll19rhWabB9b_WnoR8Q0W_w';return true;">http://www=
..open-std.org/jtc1/<wbr>sc22/wg21/docs/papers/2016/<wbr>p0273r0.pdf</a>).
<br>>
<br>> --
<br>> You received this message because you are subscribed to the Google=
Groups
<br>> "ISO C++ Standard - Future Proposals" group.
<br>> To unsubscribe from this group and stop receiving emails from it, =
send an
<br>> email to <a href=3D"javascript:" target=3D"_blank" gdf-obfuscated-=
mailto=3D"rmB2AHDSAwAJ" rel=3D"nofollow" onmousedown=3D"this.href=3D'ja=
vascript:';return true;" onclick=3D"this.href=3D'javascript:';r=
eturn true;">std-proposal...@<wbr>isocpp.org</a>.
<br>> To post to this group, send email to <a href=3D"javascript:" targe=
t=3D"_blank" gdf-obfuscated-mailto=3D"rmB2AHDSAwAJ" rel=3D"nofollow" onmous=
edown=3D"this.href=3D'javascript:';return true;" onclick=3D"this.hr=
ef=3D'javascript:';return true;">std-pr...@isocpp.org</a>.
<br>> To view this discussion on the web visit
<br>> <a href=3D"https://groups.google.com/a/isocpp.org/d/msgid/std-prop=
osals/d30abb01-3e49-424a-a410-556b60b1b833%40isocpp.org" target=3D"_blank" =
rel=3D"nofollow" onmousedown=3D"this.href=3D'https://groups.google.com/=
a/isocpp.org/d/msgid/std-proposals/d30abb01-3e49-424a-a410-556b60b1b833%40i=
socpp.org';return true;" onclick=3D"this.href=3D'https://groups.goo=
gle.com/a/isocpp.org/d/msgid/std-proposals/d30abb01-3e49-424a-a410-556b60b1=
b833%40isocpp.org';return true;">https://groups.google.com/a/<wbr>isocp=
p.org/d/msgid/std-<wbr>proposals/d30abb01-3e49-424a-<wbr>a410-556b60b1b833%=
40isocpp.org</a><wbr>.
<br></blockquote></div>
<p></p>
-- <br />
You received this message because you are subscribed to the Google Groups &=
quot;ISO C++ Standard - Future Proposals" group.<br />
To unsubscribe from this group and stop receiving emails from it, send an e=
mail to <a href=3D"mailto:std-proposals+unsubscribe@isocpp.org">std-proposa=
ls+unsubscribe@isocpp.org</a>.<br />
To post to this group, send email to <a href=3D"mailto:std-proposals@isocpp=
..org">std-proposals@isocpp.org</a>.<br />
To view this discussion on the web visit <a href=3D"https://groups.google.c=
om/a/isocpp.org/d/msgid/std-proposals/bc32cd95-7011-479c-85eb-a4ebe1777cd8%=
40isocpp.org?utm_medium=3Demail&utm_source=3Dfooter">https://groups.google.=
com/a/isocpp.org/d/msgid/std-proposals/bc32cd95-7011-479c-85eb-a4ebe1777cd8=
%40isocpp.org</a>.<br />
------=_Part_150_135361035.1456759936521--
------=_Part_149_65210392.1456759936521--
.
Author: Richard Smith <richard@metafoo.co.uk>
Date: Wed, 2 Mar 2016 12:40:15 -0800
Raw View
On Mon, Feb 29, 2016 at 7:30 AM, Abyx <fl.lllj@gmail.com> wrote:
> Let's clarify how we're going to use modules. There is an opinion that
> modules will completely replace header files.
> Many code bases have header per class, e.g. widget.h, widget.cpp, button.h,
> button.cpp, label.h, label.cpp
> And because module exports can be only in a single file, that means we are
> going to have (sub)module per class, e.g. modules widget, label, button,
> etc.
>
> Not let's look at some decent mature code base, e.g. chromium -
> https://code.google.com/p/chromium/codesearch#chromium/src/net/http/http_stream_factory_impl_job.cc
> This file has around 50 fine-grained include directives. With modules it
> will be 50 imports.
> Some files have more than hundred includes, most of the files have more than
> ten includes.
>
> No one in their right mind will copy all those dependencies into a build
> script.
> That rules out (1).
That indicates you've made your modules far too small. Your widget,
button, label, ... should probably form a single module; we don't need
fine-grained includes and imports any more; that will make build
performance *worse*, not better.
If your set of UI widget classes instead forms a single module
(perhaps with partitions for the individual classes, so you can still
partition them into files as you see fit -- see p0273r0), it is not
unreasonable for your build rules to say "I depend on that UI
library".
> 2) Build tool cannot scan for includes by itself. For imports it needs full
> preprocessing (because of #ifdef and import statements in included files).
That is the same problem that build tools see today with #include
scanning, and yet it actually does work well in practice.
> In order to do so it needs all the defines, include paths and implicit
> (forced) includes from compiler-specific environment variables and global
> (system) configuration files.
> The ninja build tool has a nice explanation of this process -
> https://ninja-build.org/manual.html#ref_headers
That's describing a different process whereby the compiler generates
dependencies as a side effect. That is not the include scanning
process I was referring to. Please refer to the build tools I listed:
http://scons.org/doc/1.1.0/HTML/scons-user/c3742.html
http://www.boost.org/doc/libs/1_43_0/doc/html/jam/usage.html#jam.usage.operation.binding.headerscan
https://github.com/bazelbuild/bazel/blob/master/src/main/java/com/google/devtools/build/lib/rules/cpp/IncludeScannable.java
> And (3) would just embed a build tool into a compiler - we take a root .cpp
> file which imports all the other modules, pass it to a compiler, and
> compiler builds all the other TU.
> That would be great except that now it's impossible to provide custom
> compiler options to certain TU.
Yes, that's true. It works for simple cases, but you'd want to pick a
different strategy for more complex builds. This is the "I just want a
simple Makefile that doesn't explicitly list any dependencies"
approach.
> On Monday, February 29, 2016 at 4:39:03 PM UTC+3, Richard Smith wrote:
>>
>> On Sun, Feb 28, 2016 at 5:41 PM, Sean Middleditch
>> <sean.mid...@gmail.com> wrote:
>> > I'll start off by saying that I don't think that Abyx's worries are a
>> > significant problem. Yes, build systems will need to change, but they've
>> > needed to change for 20 years now. If modules are the impetus for the
>> > community to finally make building C++ not ridiculous, that's just fine
>> > with
>> > me. I'm actually really hoping that modules are the final catalyst for
>> > the
>> > community to realize that CMake is this generations' Autotools. ;)
>> >
>> > That said, I do think it's worthwhile that folks at least understand
>> > where
>> > Abyx is coming from.
>> >
>> > On Friday, February 26, 2016 at 11:57:44 AM UTC-8, Richard Smith wrote:
>> >>
>> >> On Fri, Feb 26, 2016 at 7:23 AM, Abyx <fl....@gmail.com> wrote:
>> >
>> >
>> >> 1) You manually list the dependencies between your libraries in your
>> >>
>> >> build rules. This is often a good thing for your code health, but may
>> >>
>> >> not be right for everyone.
>> >
>> >
>> > We're not talking about libraries! We're talking about translation units
>> > and
>> > modules.
>>
>> We're talking about exactly the things that you configure in your
>> build tool as a library -- that is, a small number of header files
>> plus their corresponding source files. (These are not necessarily the
>> same size as .so / .dll files / executables, which would typically be
>> composed of a number of these build-system-level libraries.)
>>
>> > A single library could be made up of dozens and dozens (if not hundreds)
>> > of
>> > TUs and modules.
>> >
>> >>
>> >> 2) Your build process scans for import statements. This is not really
>> >> any harder than scanning for #includes; no real parsing is required,
>> >> at most, you need tokenize and maybe preprocess (if you want 'import
>> >> MACRO_NAME' to work). The same is true for #includes.
>> >
>> >
>> > This is much harder than scanning for #include files. No good modern C++
>> > build chain actually has a separate "scan for includes" step anymore. :)
>>
>> The only way that's true is if you define "good modern" to make it
>> true. Scons, Jam, Bazel, ... all support include scanning.
>>
>> > With headers, the dependency collection is just part of building. MSC,
>> > GCC,
>> > etc. all have flags that allow them to generate dependency chains while
>> > compiling, e.g. -MM -MF <file> on GCC. The order of compilation of TUs
>> > is
>> > usually irrelevant; in fact, it's safe to build a dependent without even
>> > knowing the dependency exists, so long as all the dependee files already
>> > exist. The dependencies are used only for checking whether files are out
>> > of
>> > date relative to another file.
>>
>> That approach does not work for large-scale distributed builds where
>> the set of files that are inputs to the build must be known before the
>> build action begins (so they can be staged to the build farm).
>>
>> > Note in particular that this also means that with modules we've just
>> > serialized a previously parallel process, which is theoretically a build
>> > deoptimization - seemingly the exact opposite of what many of us are
>> > expecting or want out of modules!
>>
>> Yes, that's theoretically true, but does not match our experience in
>> practice when building large existing codebases with (Clang's
>> implementation of) modules. While use of modules does cause there to
>> be a longer dependency chain for builds (compilation of Z.cpp depends
>> on Y1.pcm, Y2.pcm, ... already having been compiled, which depend on
>> X1.pcm, X2.pcm, ... already having been compiled, ...), and the total
>> amount of work required is somewhat higher than compiling Z.cpp
>> directly due to the serialization / deserialization overhead, those
>> individual steps are parallelizable, the work to build them is reused,
>> and despite the longer dependency chain we see significant build time
>> reductions even for clean builds.
>>
>> > That said, your option 3 is the obvious solution. It's roughly what most
>> > other modern languages with module systems do, and C++ implementations
>> > and
>> > the build tools we use are going to have to modernize to adopt to a
>> > world
>> > where the compiler plays a more integral role in the build chain than it
>> > did
>> > before.
>>
>> The downside is that option 3 parallelizes extremely poorly. If you
>> want module builds to be parallelized, distributed, and each only
>> performed once, you should look at a different option. If you only
>> care about local builds, we've found that it can be effective.
>>
>> >> 3) You ask your compiler to implicitly build (and cache) module
>> >> interfaces on demand. This requires that your compiler has some way to
>> >> map from an imported module name to the relevant interface file(s).
>> >> That could happen via some implementation-defined means (such as
>> >> Clang's module map files) or by making the module names directly
>> >> correspond to module interface files (as suggested in
>> >> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0273r0.pdf).
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "ISO C++ Standard - Future Proposals" group.
>> > To unsubscribe from this group and stop receiving emails from it, send
>> > an
>> > email to std-proposal...@isocpp.org.
>> > To post to this group, send email to std-pr...@isocpp.org.
>> > To view this discussion on the web visit
>> >
>> > https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/d30abb01-3e49-424a-a410-556b60b1b833%40isocpp.org.
>
> --
> You received this message because you are subscribed to the Google Groups
> "ISO C++ Standard - Future Proposals" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to std-proposals+unsubscribe@isocpp.org.
> To post to this group, send email to std-proposals@isocpp.org.
> To view this discussion on the web visit
> https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/d60dca15-396e-4886-92bf-3c9b79bbb829%40isocpp.org.
--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/CAOfiQqm7AZxY2%2BwcQG8SCmFDPc7TYSGX3-FWH5cuifQ3bf9kWw%40mail.gmail.com.
.
Author: Magnus Fromreide <magfr@lysator.liu.se>
Date: Thu, 3 Mar 2016 00:15:27 +0100
Raw View
On Sun, Feb 28, 2016 at 05:41:06PM -0800, Sean Middleditch wrote:
>
> That said, your option 3 is the obvious solution. It's roughly what most
> other modern languages with module systems do, and C++ implementations and
> the build tools we use are going to have to modernize to adopt to a world
> where the compiler plays a more integral role in the build chain than it
> did before.
>
How does this handle cases where the .cpp file is an intermediary on the
build path and the real source file is some .yy or .ll or something file
that have been preprocessed by some other tool?
/MF
>
> >
> > 3) You ask your compiler to implicitly build (and cache) module
> > interfaces on demand. This requires that your compiler has some way to
> > map from an imported module name to the relevant interface file(s).
> > That could happen via some implementation-defined means (such as
> > Clang's module map files) or by making the module names directly
> > correspond to module interface files (as suggested in
> > http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2016/p0273r0.pdf).
>
> --
> You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
> To post to this group, send email to std-proposals@isocpp.org.
> To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/d30abb01-3e49-424a-a410-556b60b1b833%40isocpp.org.
--
You received this message because you are subscribed to the Google Groups "ISO C++ Standard - Future Proposals" group.
To unsubscribe from this group and stop receiving emails from it, send an email to std-proposals+unsubscribe@isocpp.org.
To post to this group, send email to std-proposals@isocpp.org.
To view this discussion on the web visit https://groups.google.com/a/isocpp.org/d/msgid/std-proposals/20160302231527.GA30046%40noemi.bahnhof.se.
.