Using some form of sandbox with Python modules is a long standing best
practice with the language. There are a number of ways to have a Python
sandbox be created. At this point in time, it seems the Python community
is moving towards using the "venv" module provided with Python rather
than a separate tool. To match that we make the following changes:
- Refer to a "Python sandbox" rather than virtualenv in comments, etc.
- Install the python3-venv module in our container and not virtualenv.
- In our CI files, invoke "python -m venv" rather than "virtualenv".
- In documentation, tell users to install python3-venv and not
virtualenv.
Signed-off-by: Tom Rini <trini@konsulko.com>
With the switch to using GCC 14.2.0 in commit 001bac5f16 ("Dockerfile:
Update to gcc-14.2.0 and clang-18") in CI, we should make buildman match
this.
Signed-off-by: Tom Rini <trini@konsulko.com>
This file has about 40 pylint warnings, but no errors.
Quite a few of these warnings have been there for a while, but most are
coming from newer versions of pylint, where people come up with new
warnings.
The f-string warning is the most common one:
C0209: Formatting a regular string which could be an f-string
That feature was not available when the code was written, but it is
often more convenient than using % with a list of arguments.
This patches reduces the number of warnings in this file, with 7 left
remaining.
Signed-off-by: Simon Glass <sjg@chromium.org>
Add a helper to avoid needing to use a list within a list for this
simple case.
Update existing users of runpipe() to use this where possible.
Signed-off-by: Simon Glass <sjg@chromium.org>
Recent versions of this module call time.perf_counter() so add a patch
for this also.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Tom Rini <trini@konsulko.com>
This newer pylint produces errors about variables possibly being used
before being set. Adjust the code to pass these checks.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Tom Rini <trini@konsulko.com>
Use the "pipreqs" tool to re-create these files, with a few manual
corrections. We still need to include pytest-xdist which the tool does
not detect. We also for now don't upgrade most of the required tools as
that creates problems with various tests, which should be resolved
independently.
Signed-off-by: Tom Rini <trini@konsulko.com>
Buildman has always treated the lack of a toolchain as an infrastructure
problem rather than a build failure.
However the logic for this is not correct, since it does not write a
'done' file in this case.
As a result, one of two things can happen.
1. If a previous build ran in the same (output) directory, the outcome
of *that* build is recorded as the outcome of this one
2. Otherwise, no outcome is recorded
Obviously this inconsistency is not ideal. While (2) is rare, it can be
very confusing as the build sort-of fails but does not produce any
summary output with 'buildman -s'
Overall it seems better to attribute a toolchain issue to the boards
that it affects. This results in clear failures which can be examined,
no matter what happened in the .bm-work directory previously.
So write a 'done' file for each build when a toolchain is missing.
The end result of this patch is to make missing toolchains much more
obvious. It should be things a bit easier for novice users.
Signed-off-by: Simon Glass <sjg@chromium.org>
This is not actually used but its presence suggests that it is the
filename for the board database. Drop it to avoid confusion.
Signed-off-by: Simon Glass <sjg@chromium.org>
This is used by some boards in U-Boot and is a convenient way to deal
with common settings where using a Kconfig files is not desirable.
Detect #include files and process them as if they were part of the
original file.
Signed-off-by: Simon Glass <sjg@chromium.org>
Fixes: https://source.denx.de/u-boot/custodians/u-boot-dm/-/issues/30
This code is tested by test_scan_defconfigs() but it is useful to have
some specific tests for the KconfigScanner's operation in U-Boot. Add
a test which checks that the values are obtained correctly.
Signed-off-by: Simon Glass <sjg@chromium.org>
For most boards, the device-tree compiler is built in-tree, ignoring the
system version. Add a special option to skip this build. This can be
useful when the system dtc is up-to-date, as it speeds up the build.
Signed-off-by: Simon Glass <sjg@chromium.org>
Buildman retries a failed build when processing a branch, but does not
do this when building current source. It is useful to do this retry in
both cases, so add the logic for it.
Signed-off-by: Simon Glass <sjg@chromium.org>
When building for sandbox, self.cross is empty.
In MakeEnvironment(), CROSS_COMPILE is defined to be self.cross (with
or without a full path), optionally prefixed by the toolchain wrapper
defined in ~/.buildman. This is fine when self.cross is not empty, but
it doesn't make sense when it is:
- Either there is no wrapper and we end up with an empty CROSS_COMPILE
which is the same as not defining it (the host compiler will be used),
- Or there is a wrapper and CROSS_COMPILE will contain only the wrapper
which obviously is not a valid compiler, hence an error.
Test case:
$ sudo apt install ccache
$ grep -q toolchain-wrapper ~/.buildman || \
printf "[toolchain-wrapper]\nwrapper = ccache\n" >>~/.buildman
$ make mrproper
$ ./tools/buildman/buildman sandbox_noinst
$ ./tools/buildman/buildman sandbox_noinst
Building current source for 1 boards (1 thread, 24 jobs per thread)
sandbox: + sandbox_noinst
+arch/sandbox/lib/reloc_sandbox_efi.c:10:15: error: operator '==' has no left operand
+ 10 | #if HOST_ARCH == HOST_ARCH_X86_64
+ | ^~
[...]
The GetEnvArgs function is modified too, since the VAR_CROSS_COMPILE
case has the same issue.
In tools/buildman/test.py, testGetEnvArgs is extended and
testMakeEnvironment is added. They check the 'arm' and 'sandbox'
toolchains, with and without a wrapper.
Signed-off-by: Jerome Forissier <jerome.forissier@linaro.org>
Reviewed-by: Simon Glass <sjg@chromium.org>
The Python virtualenv tool sets up a few things in the environment,
putting its path first in the PATH environment variable and setting up
a sys.prefix different from the sys.base_prefix value.
At present buildman puts the toolchain path first in PATH so that it can
be found easily during the build. For sandbox this causes problems since
/usr/bin/gcc (for example) results in '/usr/bin' being prepended to the
PATH variable. As a result, the venv is partially disabled.
The result is that sandbox builds within a venv ignore the venv, e.g.
when looking for packages.
Correct this by detecting the venv and adding the toolchain path after
the venv path.
Signed-off-by: Simon Glass <sjg@chromium.org>
The Linux kernel documentation has transitioned from using
kconfig-language.txt to kconfig-language.rst. Therefore update all
occurrences of kconfig-language.txt.
Signed-off-by: Hiago De Franco <hiago.franco@toradex.com>
Newer versions of filelock use time.monotonic() instead of time.time().
Update the test the handle this.
It would be better if filelock had support for writing unit tests which
use locking.
Signed-off-by: Simon Glass <sjg@chromium.org>
When -x is used, buildman does not show the list of boards that will be
built, since there are no terms which cause boards to be added, only
terms which cause them to be removed.
Add a special case to fix this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Tested-by: Tom Rini <trini@konsulko.com>
There are operations in buildman that result in running the cross-tools
(such as performing size checks) and now that we have not modified PATH
to know where our tools are, these operations fail.
This reverts commit 6c0a3cf75f.
Signed-off-by: Tom Rini <trini@konsulko.com>
This is used by some Binman entry types, so add it to allow more tests
to pass.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
Add this package so we can run code-coverage tests for Binman.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reviewed-by: Tom Rini <trini@konsulko.com>
Buildman uses all available CPUs by default, so running more than one or
two concurrent processes is not normally useful.
However in some CI cases we want to be able to run several jobs at once
to save time. For example, in a lab situation we may want to run a test
on 20 boards at a time, since only the build step actually takes much
CPU.
Add an option which allows such a limit. When buildman starts up, it
waits until the number of running processes goes below the limit, then
claims a spot in the list. The list is maintained with a temporary file.
Note that the temp file is user-specific, since it is hard to create a
locked temporary file which can be accessed by any user. In most cases,
only one user is running jobs on a machine, so this should not matter.
Signed-off-by: Simon Glass <sjg@chromium.org>
When a file is removed by a commit (e.g. include/common.h yay!) it can
cause incremental build failures since one of the dependency files from
a previous build may mention the file.
Add an option to run 'make mrproper' automatically when a build fails.
This can be used to automatically resolve the problem, without always
adding the large overhead of 'make mrproper' to every build.
Signed-off-by: Simon Glass <sjg@chromium.org>
When this flag is enabled, 'make mrproper' is always used when
reconfiguring, so there is no point in doing it again.
Update this.
Signed-off-by: Simon Glass <sjg@chromium.org>
Use yamllint for checking whether YAML configuration files are adhering
to default yamllint rules.
Signed-off-by: Neha Malcom Francis <n-francis@ti.com>
Suggested-by: Nishanth Menon <nm@ti.com>
A new release has been done with this version, so update it. Use the
version numbers in dependencies also.
Signed-off-by: Simon Glass <sjg@chromium.org>
We cannot be sure that the new config is consistent, particularly when
changing a major item like CONFIG_CMDLINE. Use 'make oldconfig' to
check that and avoid any such problems.
Signed-off-by: Simon Glass <sjg@chromium.org>
When symbols switch between the inited data section and the read-only
data section their visbility changes, at present, with the -B option.
This is confusing, since adding 'const' to a variable declaration can
make it look like a significant improvement in bloat. But in fact
nothing has changed.
Add 'r' to the list of symbols types that are recorded, to correct this
problem. Add a constant to make it easier to find this code next time.
Signed-off-by: Simon Glass <sjg@chromium.org>
Reported-by: Tom Rini <trini@konsulko.com>
Reviewed-by: Tom Rini <trini@konsulko.com>
