Converting Bertini2 to a unity build
this follows templates part3
May 8, 2017
Having failed utterly at using extern template
to reduce compile time, I am attempting to use a "unity" build (master build, etc) to improve compile time.
The first step was to manually produce a unity.cpp
file which #include
s all the .cpp
files for the core library. No problem, < 3 mins manual effort on that one.
// this file includes all cpp files for this project, and represents the Unity style build.
#include "src/basics/limbo.cpp"
#include "src/basics/mpfr_complex.cpp"
#include "src/basics/mpfr_extensions.cpp"
#include "src/bertini2/bertini.cpp"
#include "src/function_tree/node.cpp"
#include "src/function_tree/operators/arithmetic.cpp"
#include "src/function_tree/operators/operator.cpp"
#include "src/function_tree/operators/trig.cpp"
#include "src/function_tree/roots/function.cpp"
#include "src/function_tree/roots/jacobian.cpp"
#include "src/function_tree/symbols/differential.cpp"
#include "src/function_tree/symbols/number.cpp"
#include "src/function_tree/symbols/special_number.cpp"
#include "src/function_tree/symbols/symbol.cpp"
#include "src/function_tree/symbols/variable.cpp"
#include "src/parallel/initialize_finalize.cpp"
#include "src/parallel/parallel.cpp"
#include "src/system/precon.cpp"
#include "src/system/slice.cpp"
#include "src/system/start_base.cpp"
#include "src/system/system.cpp"
#include "src/system/start/total_degree.cpp"
#include "src/system/start/mhom.cpp"
#include "src/system/start/user.cpp"
#include "src/tracking/explicit_predictors.cpp"
// this file should be dynamically generated, but that's a step for another day
Then, to make a new Makemodule.am
file (name arbitrary) to describe how to build libbertini2.la
with the unity source file.
#this is src/unity/Makemodule.am
# mostly copy-pasta from my original Makemodule for bertini's core.
lib_LTLIBRARIES += libbertini2.la
unity_build_source_files = \
src/unity/unity.cpp
#yep, just that one file
libbertini2_la_SOURCES = \
$(unity_build_source_files) \
$(all_headers)
#make explicit the dependence on all header files. this should be adjusted to be tighter
rootinclude_HEADERS += include/bertini2/bertini.hpp
# depends on a lot of libraries
libbertini2_la_LIBADD = $(BOOST_LDFLAGS) \
$(BOOST_FILESYSTEM_LIB) $(BOOST_SYSTEM_LIB) $(BOOST_CHRONO_LIB) $(BOOST_REGEX_LIB) $(BOOST_TIMER_LIB) $(MPI_CXXLDFLAGS) $(BOOST_UNIT_TEST_FRAMEWORK_LIB) $(BOOST_SERIALIZATION_LIB) $(BOOST_LOG_LIB) $(BOOST_LOG_SETUP_LIB) $(BOOST_THREAD_LIB)
bin_PROGRAMS += bertini2
bertini2_SOURCES = \
src/bertini2/bertini.cpp \
include/bertini2/blackbox/main_mode_switch.hpp src/blackbox/main_mode_switch.cpp \
include/bertini2/blackbox/argc_argv.hpp src/blackbox/argc_argv.cpp \
include/bertini2/parallel.hpp \
include/bertini2/parallel/initialize_finalize.hpp src/parallel/initialize_finalize.cpp
bertini2_LDADD = $(BOOST_LDFLAGS) $(BOOST_FILESYSTEM_LIB) $(BOOST_SYSTEM_LIB) $(BOOST_CHRONO_LIB) $(BOOST_REGEX_LIB) $(BOOST_TIMER_LIB) $(MPI_CXXLDFLAGS) $(BOOST_SERIALIZATION_LIB) $(BOOST_LOG_LIB) $(BOOST_LOG_SETUP_LIB) $(BOOST_THREAD_LIB) libbertini2.la
bertini2_CXXFLAGS = $(BOOST_CPPFLAGS)
Results
Unity has been implemented, let's test it out! I compiled with -O2
in Clang coming with XCode 8.
Unity, single thread: 1m15s. Incremental, single thread: 4m08s.
30% of original time, saving 70%. This is just for the core
library and nearly-trivial blackbox
program, and not including building the tests (I only made up a unity.cpp
file for the core).
Since my little virtual build server farm (Thanks CRC!!!) consists of virtual machines without a lot of CPU's, this may prove beneficial. I do, however, need to make sure that the memory usage doesn't go crazy. Sometimes my .cpp
files can take a ton of ram, due to the extensive use of templating...
Conclusions
I am pretty happy with the "unity build" thing. It appears to have lead to a 70% reduction in single-thread compile time for the core.
A last note: multithreaded compilation was still faster... Because I can use 8 threads on my Mac. So, this style of build should remain an option, not a requirement. I will add a --with-unity_build
option to the configure.ac
file, so a user can choose to use it. A last question: which to make default, unity or incremental?