When 'opfw_resource' target is used as a dependency, it seems that make
can't determine when the resource file is remade, and sometimes may
start building GCS before the resource really exists. This is an attempt
to fix that issue by referencing the real file instead of phony target.
This allows to define OPENPILOT_DL_DIR and OPENPILOT_TOOLS_DIR to override
local tools download and installation directory. So the same toolchains
can be used for all working copies. Particularly useful for CI server
agents, but also for local installations.
Paths should use forward (unix-style) slashes.
If no OPENPILOT_* variables found, makefile internal DL_DIR and TOOLS_DIR
are used. They still can be overriden by the make command line parameters.
This changeset:
- moves all ground targets one level up, under build directory. The
build/ground was created as a workaround, now unnecessary;
- fixes QtCreator builds, they are separated from command line builds;
- moves GCS autogenerated files into openpilotgcs-synthetics directory.
The resulting build subdirectory now looks like:
build
openpilotgcs <- Qt-Creator build directory
openpilotgcs-synthetics <- version-info and opfw_resource
openpilotgcs_debug
openpilotgcs_release <- Makefile build directory
uavobject-synthetics
uavobjgenerator
NOTE: you should update the shadow build path in QtCreator to build,
not build/ground as before.
+review OPReview
This is the first cleanup pass through makefiles and pios.
Probably it is difficult to track changes due to the nature of them.
I would recommend to look at resulting files and compiled code instead.
NOTE: original branch was rebased and lot of conflicts were fixed on
the way. So do not expect that every commit in this series will be
buildable (unlike original branch). Only final result was tested.
The main goal was to remove as much duplication of code (and copy/paste
errors) as possible, moving common parts out of Makefiles. It still is
not perfect, and mostly no code changes made - Makefiles and #ifdefs only.
But please while testing make sure that all code works as before, and no
modules/options are missed by accident.
Brief list of changes:
- Moved common parts of Makefiles into the set of *.mk files.
- Changed method of passing common vars from top Makefile to lower ones.
- Some pios cleanup, mostly #ifdefs, and all pios_config.h files.
- Many obsolete files removed (for instance, AHRS files, op_config.h).
- Many obsolete or unused macros removed or fixed/renamed (ALL_DIGNOSTICS).
- Unified pios_config.h template. Please don't remove lines for board
configs, only comment/uncomment them. Adding new PIOS options, please
propagate them to all board files keeping the same order.
- Some formatting, spacing, indentation (no line endings change yet).
- Some cosmetic fixes (no more C:\X\Y\filename.c printings on Windows).
- Added some library.mk files to move libs into AR achives later.
- EntireFlash target now uses cross-platform python script to generate bin
files. So it works on all supported platforms: Linux, OSX, Windows.
- Top level packaging is completely rewritten. Now it is a part of top
Makefile. As such, all dependencies are checked and accounted, no
more 'make -j' problems should occur.
- Default GCS_BUILD_CONF is release now, may be changed if necessary
using 'make GCS_BUILD_CONF=debug gcs'.
- GCS build paths are separated into debug and release, so no more obj
file clashes. Packaging system supports only release builds.
- New target is introduced: 'clean_package'. Now 'make package' does not
clean build directory. Use clean_package instead for distributable builds.
- Targets like 'all', 'opfw_resource', etc now will print extra contex
in parallel builds too.
- If any of 'package', 'clean_package', 'opfw_resource' targets are given
on command line, GCS build will depend on the resource, so all fw_*.opfw
targets will be built and embedded into GCS. By default GCS does not
depend on resource, and will be built w/o firmware (unless the resource
files already exist and the Qt resource file is generated).
- fw_simposix (ELF executable) is now packaged for linux. Run'n'play!
- Make help is refined and is now up to date.
Still broken:
- UnitTests, should be fixed
- SimPosix: buildable, but should be reworked.
Next planned passes to do:
- toolchain bootstrapping and packaging (including windows - WIP)
- CMSIS/StdPeriph lib cleanup
- more PIOS cleanup
- move libs into AR archives to save build time
- sim targets refactir and cleanup
- move android-related directories under <top>/android
- unit test targets fix
- source code line ending changes (there are many different, were not changed)
- coding style
Merging this, please use --no-ff git option to make it the real commit point
Conflicts:
A lot of... :-)
opfw_resource make target now depends on built firmware images. They are
referenced directly from Qt resource file generated. No extra copies
are now necessary.
Makefile uses toprel function to display nice short relative paths when
building. But different ways of getting current path (using . or pwd
or MAKEFILE_LIST, etc) give different cases for Windows drive letters.
As result, some paths are not translated by toprel function and look
not nice. Passing known TOPDIR value to sub-make fixes this:
...
CC flight/Modules/FirmwareIAP/firmwareiap.c
CC flight/Modules/Telemetry/telemetry.c
CC flight/Modules/System/systemmod.c
CC d:/Work/OpenPilot/toolchains/flight/targets/CopterControl/System/coptercontrol.c
CC d:/Work/OpenPilot/toolchains/flight/targets/CopterControl/System/pios_board.c
...
Instead of
...
CC flight/Modules/FirmwareIAP/firmwareiap.c
CC flight/Modules/Telemetry/telemetry.c
CC flight/Modules/System/systemmod.c
CC flight/targets/CopterControl/System/coptercontrol.c
CC flight/targets/CopterControl/System/pios_board.c
...
Make targets changed to:
make ut_<test>
make ut_<test>_tap
make ut_<test>_run
make ut_<test>_clean
make all_ut
make all_ut_tap
make all_ut_run
make all_ut_clean
Conflicts:
Makefile
When running the build as root, any bugs in the Makefiles
can have disasterous results. This will be even more
important once we start automatically running unit tests
as part of our build.
This change ensures that the builds cannot be run as root.
Build it with:
make ut_logfs
Create a raw flash file like this:
dd if=/dev/zero bs=1 count=2MiB | tr '\000' '\377' > theflash.bin
Run it with:
./build/unit_tests/logfs/logfs.elf && echo 'all good'
Conflicts:
Makefile
This removes the detailed output of the GCS build by
default so that warnings and errors become more evident.
Full output can be enabled by adding "V=1" to the make
command line. Same mechanism as is used for the firmware
builds.
Looks like mac doesn't support the -r option to sed to
turn on extended regexps. It only supports the -E option
which linux doesn't.
So, simplest fix is to avoid extended regexps and just
use as many dots as possible from now on.
Windows doesn't like -I. Mac doesn't like -i.
This now avoids using either option to xargs by
using awk to reorder the ln command line args prior
to invoking xargs.
Use '-i' (deprecated) in xargs instead of '-I' so old
versions of xargs work.
Use only relative paths to files passed to tar since the
windows version doesn't seem to handle 'c:/' at the start
of the path.
QT SDK version was changed but the install dir wasn't changed
to match. This makes it impossible to keep both versions
installed at the same time which is essential in order to work
on/build old and new branches.
This update gives the new SDK version a unique install dir so
it can coexist with older versions.
Extra message context is also useful even when building more
than one target in the same invocation of make.
This example would enable extra context:
make fw_coptercontrol bl_coptercontrol
When building the various all_* targets, it was hard to tell which
board/build-type that each line of output applied to. Now, the
all_* target types will include something like:
CC [fw|cc ] flight/PiOS/STM32F10x/pios_gpio.c
which includes the necessary additional context.
This will help with identifying the context for warnings and errors
when building a group of targets.
Conflicts:
Makefile
The simposix build doesn't work on Windows or Mac so it shouldn't
be in the all_flight build target that is used as a pre-submit
check on all platforms.
Extra message context is also useful even when building more
than one target in the same invocation of make.
This example would enable extra context:
make fw_coptercontrol bl_coptercontrol
When building the various all_* targets, it was hard to tell which
board/build-type that each line of output applied to. Now, the
all_* target types will include something like:
CC [fw|cc ] flight/PiOS/STM32F10x/pios_gpio.c
which includes the necessary additional context.
This will help with identifying the context for warnings and errors
when building a group of targets.
Extra message context is also useful even when building more
than one target in the same invocation of make.
This example would enable extra context:
make fw_coptercontrol bl_coptercontrol
When building the various all_* targets, it was hard to tell which
board/build-type that each line of output applied to. Now, the
all_* target types will include something like:
CC [fw|cc ] flight/PiOS/STM32F10x/pios_gpio.c
which includes the necessary additional context.
This will help with identifying the context for warnings and errors
when building a group of targets.
The simposix build doesn't work on Windows or Mac so it shouldn't
be in the all_flight build target that is used as a pre-submit
check on all platforms.
Now that we have a USB descriptor for HID+VCP that works
on Ubuntu 11.10, MacOS 10.7.3, Win7-SP1-32bit,
WinXP-SP3-32bit, Win7-SP1-64bit.
VCP is known to not work on WinXP without service pack 3.
Bootloader updaters were all broken due to recent
changes in LED configuration handling. Removing
LED support from the BU loads allows them to build
again, but they provide no feedback about when they're
finished writing flash.
Bootloader updaters were all broken due to recent
changes in LED configuration handling. Removing
LED support from the BU loads allows them to build
again, but they provide no feedback about when they're
finished writing flash.
Qt 4.8+ is now required to build the GCS.
This version of the SDK installer has dropped support
for the --installdir option so users will need to manually
specify the path for installation as:
<top>/tools/qtsdk-v1.2
in order for the Makefile to pick up this toolchain automatically.
This update drops support for Ubuntu 9.10 (Karmic) due to an
incompatibility with the glibc that ships with that release.
Support should still be possible with Ubuntu 10.04 (Lucid) and
newer.
Qt 4.8+ is now required to build the GCS.
This version of the SDK installer has dropped support
for the --installdir option so users will need to manually
specify the path for installation as:
<top>/tools/qtsdk-v1.2
in order for the Makefile to pick up this toolchain automatically.
This update drops support for Ubuntu 9.10 (Karmic) due to an
incompatibility with the glibc that ships with that release.
Support should still be possible with Ubuntu 10.04 (Lucid) and
newer.
The CC and PipX bootloader updater (BU) builds don't currently
work due to some recent changes in how LEDs are handled.
Remove them from the default BU targets so that the all_flight
target can build clean again.
Also fix a linker warning in OP build.
Linux users can now cross-compile openocd to run on windows.
Here are the steps:
sudo apt-get install mingw32
make ftd2xx_install
make libusb_win_install
make openocd_git_win_install
(cd tools; zip -r openocd-win-patched.zip openocd_win/)
This should simplify rolling out new versions of the tool and
provides an example for how to cross-compile future tools as
well.
A separate makefile for EF (Entire Flash) targets allows
the removal of board-specific hard-coded addresses. These
values can now be included from the board-info.mk files
for each board.
These new build targets will construct a flash image that
includes both a bootloader as well as a firmware image all
in one. Examples:
make ef_coptercontrol
make ef_revolution
This also adds support for a new target for writing this EF
image to boards that support a DFU interface (only F4-based
boards). This can be done by running:
make dfuutil_install
# Boot board with SBL asserted to activate STM32 DFU ROM
# Attach USB cable
make ef_revolution_dfu
Bootloader images are also now padded out to occupy the full
BL flash region.
This tool can be used to read/write the interal device flash
in an STMF4 MCU via the built-in bootloader.
dfu-util can be downloaded and installed using this command:
make dfuutil_install
The homepage for this tool is at:
http://dfu-util.gnumonks.org/
Added new unbrick_<board> targets for each board.
These targets use the stm32flash utility to install
a bootloader on to the board. This is a tool of
last resort and should not be used for normal firmware
upgrades.
Usage:
make unbrick_<board> UNBRICK_TTY=/dev/ttyUSB0
Where:
<board> is one of coptercontrol, pipxtreme,
openpilot, ins, ahrs
UNBRICK_TTY is set to the serial device connected
to the board.
This tool can be used as a last resort for
installing a bootloader to a completely bricked
board that is unresponsive to even the rescue
functionality in the GCS uploader gadget.
Building all_<board> will build all available embedded
images for <board>. New targets:
- all_ahrs
- all_openpilot
- all_coptercontrol
- all_pipxtreme
- all_ins
and their associated _clean targets to remove the build output.
- New macros for fw, bl and bu rules in top-level make
- Per-board info factored into make/board/*/board-info.mk
- Per-board info now shared btw. fw, bl and blupd for each board
- BOARD_TYPE, BOARD_REVISION, BOOTLOADER_VERSION, HW_TYPE
- MCU, CHIP, BOARD, MODEL, MODEL_SUFFIX
- START_OF_BL_CODE, START_OF_FW_CODE
- blupd_* goals renamed to bu_*
- all_blupd goal renamed to all_bu
- firmware goals renamed to fw_*, board name goals are preserved
- bu_*_program now writes updater to correct address for all boards
- BL updater firmware builds now produce .opf format including
version info blob.
- BL updater firmware name now includes board name.
- INS makefile brought up to date w.r.t. linker scripts
Now 'make release' on the top level will build release package.
For Windows it will build OpenPilot GCS Installer and all firmware
binaries which are now included into the GCS distribution and will
be installed into the <path>/OpenPilot/share/firmware-<version>
directory.
For other systems it will build all firmware files and the GCS, but
latter is not packaged yet.
In preparation for release build the Windows installer source is moved
out of ground/gcs source tree into release directory. It is also disabled
in the qmake source file and removed from top level Makefile.
On Windows cd doesn't mean that current directory is as expected since
there is also current drive. So in some rare cases mkdir followed by cd
doesn't change the directory which can result in Makefile overwrites by
qmake, etc. So it is safer to replace ';' by '&&' but need to check if
it still works on Windows under cmd or from Qt-Creator.
Certain environment variables or command-line overrides
can result in difficult-to-diagnose build issues. These
variables are now filtered from the calling environment
prior to invoking the lower level makefiles.
These variables which may affect gcc are removed from the
environment and from the command-line overrides:
TMPDIR GCC_EXEC_PREFIX COMPILER_PATH LIBRARY_PATH
CFLAGS CPATH C_INCLUDE_PATH CPLUS_INCLUDE_PATH
OBJC_INCLUDE_PATH DEPENDENCIES_OUTPUT
These variables which are now deprecated are removed from
the environment and from the command-line overrides:
USE_BOOTLOADER
This change is made up of a number of tightly coupled
changes:
- Deprecate the use of the USE_BOOTLOADER command-line
option. It is now hard-coded in each Makefile.
Overriding it on the command line is not allowed.
- Split apart the memory declaration and the section
declaration in all linker files (*_memory.ld and
*_sections.ld).
- Describe the split between bootloader and app sections
of flash in each board's _memory.ld file.
- Change program target to selectively erase flash so
that the installed bootloader is preserved across even
JTAG programming operations.
- All elf files are built with debug symbols and are not
stripped. This should help debugging with gdb. The
images programmed on the boards are all .bin files now
which do not include symbols.
