NDK 10e

Automated build recipe

See https://gitlab.com/android-rebuilds/auto/tree/master/ndk-10e.

Principle of operation

NDK is build out of the standard SDK tree. It uses a few Git checkouts, and then grabs other Git repositories at build time.

Those Git repositories do not have Git tags, which complexifies the rebuild a great deal.

Recreate matching build environment

The current reference build environment for this version is Ubuntu 14.04.

Make sure 'bash' is the default shell, otherwise build-gabi++.sh will fail.

Though see below for issues with 10.04 and 12.04.

Build dependencies

# Base Android build dependencies (from https://source.android.com/source/initializing.html, section "Installing required packages (Ubuntu 12.04)")
# Dropping mingw to skip windows builds for now.
# Installing g++-multilib first otherwise apt-get complains.
apt-get install build-essential g++-multilib
apt-get install bison git ca-certificates gperf libxml2-utils make python-networkx zlib1g-dev:i386 zip
apt-get install libncurses5-dev:i386
apt-get install libncurses5-dev  # needed?

# Additional NDK build dependencies (completed from https://android.googlesource.com/platform/ndk/+/master/README.md)
apt-get install curl texinfo bison flex libtool pbzip2 groff
apt-get install autoconf automake

# Repackaging
apt-get -y install p7zip-full

Determining precise source commits

  • Max date: RELEASE.TXT is 2015-04-07T11:32:42+0000
  • r10e/build/tools/toolchain-patches/gcc/ has 12 patches, not 13 (13th added 2015-04-06T16:05:44+0800)
  • r10e/build/tools/download-toolchain-sources.sh ("gdb-7.7") references 2015-03-30T18:25:27+0000
  • r10e/build/core/build-local.mk matches commit 0de21a10c0a96c3570a283462dc213fcac373607 (2015-04-07T09:12:51+0000 before its merge a month later - wasn't pushed/gerrit'd at time of release?))
  • for development/ and bionic/ we can only guess, those are not bundled

Preparing build tree

# documented: ndk/docs/text/DEVELOPMENT.text
git clone https://android.googlesource.com/platform/ndk.git ndk
git clone https://android.googlesource.com/platform/development.git development
# undocumented: http://grokbase.com/t/gg/android-ndk/13bfdn134a/no-build-platforms-sh-script
git clone https://android.googlesource.com/platform/bionic.git bionic

(cd ndk/         && git checkout 0de21a10c0a96c3570a283462dc213fcac373607)
(cd development/ && git checkout $(git rev-list -n 1 --before='2015-04-07T11:32:42+0000' HEAD))
(cd bionic/      && git checkout $(git rev-list -n 1 --before='2015-04-07T11:32:42+0000' HEAD))

Build NDK

First grab all sources:

export NDK=~/wd/ndk
export NDK_LOGFILE=~/wd/ndk.log
bash $NDK/build/tools/download-toolchain-sources.sh --git-date='2015-04-07T11:32:42+0000' ~/wd/ndk-dl
# ndk-dl: 3.7GB

Python fix:

sed -i -e 's/LIBS="@LIBS@ $SYSLIBS -lpython${VERSION}${ABIFLAGS}"/LIBS="-lpython${VERSION}${ABIFLAGS} @LIBS@ $SYSLIBS"/' \
  ~/wd/ndk-dl/python/Python-2.7.5/Misc/python-config.sh.in
# Not sure how NDK r10e was built - this error is triggered by gcc >= 4.4 (aka >= 10.04)
# 12.04 fails because LLVM requires GCC >= 4.7 (+ other minor issues), so the reference system is well 14.04
# Fixed in 2015-08-27 (wasn't pushed/gerrit'd at time of release?)
# https://android.googlesource.com/toolchain/python/+/b7811603a7e3aba7db9453d81a677dda40cca435

Missing symlink, cf. df5abfaa906e8f6d0b58ec43d45417879f4b762a (2015-05-26, wasn't pushed/gerrit'd at time of release?)

# 7.7 requested by 'build-gdbserver.sh' using version from 'dev-defaults.sh'
ln -s gdb-7.6 $NDK/sources/android/libthread_db/gdb-7.7

-Werror issue, cf. 0b0712078f2f34effa563a8359ff2f5137ada641 Wed Jun 17 21:48:24 2015 -0700

sed -i -e 's/CONFIGURE_FLAGS=$CONFIGURE_FLAGS" --disable-inprocess-agent"/CONFIGURE_FLAGS=$CONFIGURE_FLAGS" \
  --disable-inprocess-agent --enable-werror=no"/' \
  $NDK/build/tools/build-gdbserver.sh

build-gnu-libstdc++.sh is referencing ndk/toolchains/aarch64-linux-android-4.8/ which doesn't exist since cc968d5e85f93e5cfcec7f681e7a0acd75794963 Fri Jun 6 20:23:18 2014 +0800

# Fixed in f95e30a7e03a518ee0029876d2ade75bcbdd960f  Tue Jun 9 16:53:44 2015 +0800 - post-release again.
# That's only the 4th compilation fix that was committed post-release ?!?
sed -i -e 's/&& version_is_at_least/\&\& ! version_is_at_least/' $NDK/build/tools/build-gnu-libstdc++.sh

Launch the proper build:

export NDK_TMPDIR=~/wd/ndk-tmp
mkdir -p $NDK_TMPDIR/release-r10e
time bash $NDK/build/tools/make-release.sh --toolchain-src-dir=$HOME/wd/ndk-dl --release=r10e --incremental

The GNU/Linux NDK release is in /tmp/ndk-android/release/android-ndk-r10e-linux-x86.tar.bz2 :)

Disk usage:

7.0G    /tmp/ndk-android/
8.6G    /home/android/wd/
>15.6GB Total

Repackage:

cd /tmp/ndk-android/release/
tar xf android-ndk-r10e-linux-x86.tar.bz2
#tar cJf android-ndk-r10e-linux-x86.tar.xz android-ndk-r10e/ # 20mn mono-proc; TODO: try pxz or pixz; 511MB vs. Google's 383M
7za a -sfx ~/wd/android-ndk-r10e-linux-x86.bin android-ndk-r10e/  # 10mn using ~2 procs; 347MB w/ sfx

The final GNU/Linux NDK release is in ~/wd/android-ndk-r10e-linux-x86.bin :)

TODOs

  • windows release
  • diagnose missing android-ndk-r10e/platforms/android-21/arch-arm/usr/lib/rs/
  • check if it's normal that LLVM-3.5 and GCC-3.8 were removed in master builds

Future builds

These instructions are obsoleted by https://android.googlesource.com/platform/ndk/+/f2baa4312cfd2bbbb2c983457dee3be8e3d1c5a6

Notes

aarch64 support

We noted build-gnu-libstdc++.sh is referencing ndk/toolchains/aarch64-linux-android-4.8/ which doesn't exist since cc968d5e85f93e5cfcec7f681e7a0acd75794963 Fri Jun 6 20:23:18 2014 +0800

In any case, toolchains/aarch64-linux-android-4.8 was never published, aarch64-linux-android-* is introduced with gcc-4.9 in NDK r10c.

Fixed in f95e30a7e03a518ee0029876d2ade75bcbdd960f Tue Jun 9 16:53:44 2015 +0800 - post-release again.

Interestingly:

$ find -name "libstdc++*"|xargs ls -ldrt| grep arm64
-rw-r----- 1 personnel personnel 182608 oct.  15  2014 ./platforms/android-21/arch-arm64/usr/lib/libstdc++.a
-rwxr-x--- 1 personnel personnel   6194 févr.  2  2015 ./platforms/android-21/arch-arm64/usr/lib/libstdc++.so

Looks like arch-arm64 was reused from r10c:

9ed6af0606d722020986cb227419808c  ./android-ndk-r10c/platforms/android-21/arch-arm64/usr/lib/libstdc++.a
9ed6af0606d722020986cb227419808c  ./android-ndk-r10d/platforms/android-21/arch-arm64/usr/lib/libstdc++.a
9ed6af0606d722020986cb227419808c  ./android-ndk-r10e/platforms/android-21/arch-arm64/usr/lib/libstdc++.a

References

Follow other instructions in DEVELOPMENT.html to build prebuilt toolchain binaries for your host platform and test your changes in your workspace.

Ways to detail the build

# DEVELOPMENT.text

# Section II
#bash $NDK/build/tools/gen-platforms.sh
# + CC='ERROR: arm toolchain not installed: /home/android/wd/ndk/toolchains/arm-linux-androideabi-4.6/prebuilt/linux-x86/bin/arm-linux-androideabi-gcc
# Section 2
#$NDK/build/tools/rebuild-all-prebuilt.sh
##$NDK/build/tools/rebuild-all-prebuilt.sh --mingw

# Section III.2
bash $NDK/build/tools/download-toolchain-sources.sh --package
#bash $NDK/build/tools/download-toolchain-sources.sh ~/ndk-dl/

bash $NDK/build/tools/rebuild-all-prebuilt.sh --toolchain-pkg=/tmp/android-ndk-toolchain-$(date +%XXX).tar.bz2
#bash $NDK/build/tools/rebuild-all-prebuilt.sh --toolchain-src-dir=~/ndk-dl/
#bash $NDK/build/tools/rebuild-all-prebuilt.sh --mingw --toolchain-pkg=/tmp/android-ndk-toolchain-$(date +%XXX).tar.bz2
# TODO: + --package?

cd $NDK
tar xf /tmp/ndk-prebuilt/prebuilt-$(date +%XXX)/*.tar.bz2

# Section IV
# Actually scratch the above and do:
yes | bash $NDK/build/tools/make-release.sh
# Doesn't work anymore, the download script checks the master branch,
# and the build system changed since it was written

Error messages and resolutions

#Copying x86 sysroot libs from $SRC/platforms/android-9/arch-x86/lib to $DST/platforms/android-9/arch-x86/usr/lib.
#Generating platform 9 crtbrand assembly code: /home/android/wd/ndk/platforms/android-9/arch-x86/usr/lib/crtbrand.s
#Generating x86 C runtime object: crtbegin_dynamic.o
#ERROR: Could not generate crtbegin_dynamic.o from /home/android/wd/development/ndk/platforms/android-9/arch-x86/src/crtbegin.c /home/android/wd/ndk/platforms/android-9/arch-x86/usr/lib/crtbrand.s
#ERROR: Could not generate crtbegin_dynamic.o from /home/android/wd/development/ndk/platforms/android-9/arch-x86/src/crtbegin.c /home/android/wd/ndk/platforms/android-9/arch-x86/usr/lib/crtbrand.s
#Please see the content of /tmp/ndk-android/tmp/tests/tmp-platform.log for details!
#Please see the content of /tmp/ndk-android/tmp/tests/tmp-platform.log for details!
### COMMAND: /home/android/wd/ndk/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86/bin/arm-linux-androideabi-gcc   -Wl,-shared,-Bsymbolic -Wl,-soname,libz.so -nostdlib -o /tmp/ndk-android/tmp/tests/tmp-platform.o /tmp/ndk-android/tmp/tests/tmp-platform.c -Wl,--exclude-libs,libgcc.a
#crtbegin.c:29:43: fatal error: ../../bionic/libc_init_common.h: No such file or directory
# #include "../../bionic/libc_init_common.h"
#                                            ^
#compilation terminated.
#ERROR: Could not generate platforms and samples directores!
#ERROR: Could not generate platforms and samples directores!
#ERROR: Could not build target prebuilts!
#ERROR: Could not build target prebuilts!
#ERROR: Can't build  binaries.
# (tmp-platform.log containing exactly what's below (the crtbegin.c error))
# (running the command manually works - concurrency issue?)

=> checkout bionic.git

/home/android/wd/ndk/toolchains/arm-linux-androideabi-4.8/prebuilt/linux-x86/bin/arm-linux-androideabi-gcc --sysroot=/tmp/ndk-android/build/gdbserver/sysroot -O2 -fno-short-enums    -I. -I/home/android/ndk-dl/gdb/gdb-7.7/gdb/gdbserver -I/home/android/ndk-dl/gdb/gdb-7.7/gdb/gdbserver/../common -I/home/android/ndk-dl/gdb/gdb-7.7/gdb/gdbserver/../regformats -I/home/android/ndk-dl/gdb/gdb-7.7/gdb/gdbserver/.. -I/home/android/ndk-dl/gdb/gdb-7.7/gdb/gdbserver/../../include -I/home/android/ndk-dl/gdb/gdb-7.7/gdb/gdbserver/../gnulib/import -Ibuild-gnulib-gdbserver/import -Wall -Wdeclaration-after-statement -Wpointer-arith -Wformat-nonliteral -Wno-char-subscripts -Wempty-body -Werror -DGDBSERVER -c -o gdbreplay.o -MT gdbreplay.o -MMD -MP -MF .deps/gdbreplay.Tpo /home/android/ndk-dl/gdb/gdb-7.7/gdb/gdbserver/gdbreplay.c
/home/android/ndk-dl/gdb/gdb-7.7/gdb/gdbserver/thread-db.c: In function 'thread_db_load_search':
/home/android/ndk-dl/gdb/gdb-7.7/gdb/gdbserver/thread-db.c:561:30: error: assignment from incompatible pointer type [-Werror]
   tdb->td_thr_event_enable_p = &td_thr_event_enable;
                              ^
cc1: all warnings being treated as errors
make: *** [thread-db.o] Error 1
make: *** Waiting for unfinished jobs....
Could not build arm-linux-androideabi-4.8 gdbserver. Use --verbose to see why.
Could not build arm-linux-androideabi-4.8 gdbserver. Use --verbose to see why.

=> -Werror subsenquently disabled, cf. 0b0712078f2f34effa563a8359ff2f5137ada641 Wed Jun 17 21:48:24 2015 -0700

Forcing generation of 32-bit host binaries on x86_64
HOST_ARCH=x86
HOST_TAG=linux-x86
gnustl_static gcc-4.8 arm64-v8a : configuring
## COMMAND: /home/android/ndk-dl/gcc/gcc-4.8/libstdc++-v3/configure --prefix=/tmp/ndk-android/tmp/build-15632/build-gnustl/install-arm64-v8a-4.8/ --host=aarch64-linux-android --enable-static --disable-shared --disable-libstdcxx-visibility --enable-libstdcxx-time --disable-symvers --disable-multilib --disable-nls --disable-sjlj-exceptions --disable-tls --disable-libstdcxx-pch --with-gxx-include-dir=/tmp/ndk-android/tmp/build-15632/build-gnustl/install-arm64-v8a-4.8//include/c++/4.8
configure: WARNING: If you wanted to set the --build type, don't use --host.
    If a cross compiler is detected then cross compile mode will be used.
checking build system type... x86_64-unknown-linux-gnu
checking host system type... aarch64-unknown-linux-android
checking target system type... aarch64-unknown-linux-android
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for aarch64-linux-android-strip... /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-strip
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... no
checking for mawk... mawk
checking whether make sets $(MAKE)... yes
checking for aarch64-linux-android-gcc... /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-gcc
checking for suffix of object files... configure: error: in `/tmp/ndk-android/tmp/build-15632/build-gnustl/static-arm64-v8a-4.8':
configure: error: cannot compute suffix of object files: cannot compile
See `config.log' for more details.
ERROR: Could not configure gnustl_static gcc-4.8 arm64-v8a 

configure:3559: checking for aarch64-linux-android-gcc
configure:3586: result: /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-gcc
configure:3855: checking for C compiler version
configure:3864: /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-gcc --version >&5
/home/android/ndk-dl/gcc/gcc-4.8/libstdc++-v3/configure: line 3866: /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-gcc: 
No such file or directory
configure:3875: $? = 127
configure:3864: /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-gcc -v >&5
/home/android/ndk-dl/gcc/gcc-4.8/libstdc++-v3/configure: line 3866: /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-gcc: 
No such file or directory
configure:3875: $? = 127
configure:3864: /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-gcc -V >&5
/home/android/ndk-dl/gcc/gcc-4.8/libstdc++-v3/configure: line 3866: /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-gcc: 
No such file or directory
configure:3875: $? = 127
configure:3864: /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-gcc -qversion >&5
/home/android/ndk-dl/gcc/gcc-4.8/libstdc++-v3/configure: line 3866: /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-gcc: 
No such file or directory
configure:3875: $? = 127
configure:3891: /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-gcc -o conftest -fPIC  --sysroot=/home/android/wd/ndk/platforms/android-21/arch-arm64 -fexceptions -funwind-tables -D__BIONIC__ -O2 -ffunction-sections -fdata-sections -g -mfix-cortex-a53-835769  --sysroot=/home/android/wd/ndk/platforms/android-21/arch-arm64 -lc  conftest.c  >&5
/home/android/ndk-dl/gcc/gcc-4.8/libstdc++-v3/configure: line 3892: /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-gcc: 
No such file or directory
configure:3894: $? = 127
configure:4082: checking for suffix of object files
configure:4104: /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-gcc -c -fPIC  --sysroot=/home/android/wd/ndk/platforms/android-21/arch-arm64 -fexceptions -funwind-tables -D__BIONIC__ -O2 -ffunction-sections -fdata-sections -g -mfix-cortex-a53-835769  --sysroot=/home/android/wd/ndk/platforms/android-21/arch-arm64 conftest.c >&5
/home/android/ndk-dl/gcc/gcc-4.8/libstdc++-v3/configure: line 4106: /home/android/wd/ndk/toolchains/aarch64-linux-android-4.8/prebuilt/linux-x86/bin/aarch64-linux-android-gcc: No such file or directory
configure:4108: $? = 127
configure: failed program was:

=> bug in build/tools/build-gnu-libstdc++.sh, cf. f95e30a7e03a518ee0029876d2ade75bcbdd960f

build time

Using i7-4500U (2x1.80GHz + HT), SSD disk:

  • make: 05h30mn
  • tar+7za: 30mn