Commit Graph

134 Commits

Author SHA1 Message Date
Matthias Schiffer
ee5ec5afe5 build: rewrite features.sh in Lua
The `features` file is converted to a Lua-based DSL.

A helper function `_` is used in the DSL; this will return the original
string for enabled features, and nil for disabled features. This allows
to use boolean operations on features without making the code too
verbose.

Besides having more readable and robust code, this also fixes the bug
that all files `packages/*/features` were evaluated instead of only
using the feature definitions of currently active feeds.
2020-07-01 20:35:22 +02:00
Matthias Schiffer
0fd5905fc2
build: target_config_lib: introduce concat_list helper 2020-07-01 20:10:40 +02:00
Matthias Schiffer
b3edfd292a
build: target_config_lib: do not build unused packages for targets without opkg (#2051)
Normally, we build all nonshared packages (which includes all kernel
modules) to generate an opkg feed for later package installations by
users. On targets without opkg, this just wastes time - disable it.
2020-06-11 02:16:49 +02:00
Matthias Schiffer
8b64517f1b
build: target_config_check: make check more lenient
Always allow options set to builtin (=y) when modular setting (=m) is
expected. This can happen when a package is added explicitly (in a
target defintion or site.mk) that is also pulled in as a dependency of
another builtin package.

Fixes: 9e23534ec3 ("build: rework config generation")
Fixes: #2046
2020-06-05 22:27:10 +02:00
Chrissi^ (Chris Fiege)
1956696da5
openwrt: Store Kernel Debug-Info (#1971)
This change stores a Kernel with Debug-Symbols for the current
architecture in a new output directory '<outputdir>/debug'.
This allows a developer or operator of a network to store the kernel
along with the actual images. In case of a kernel oops the debug
information can be used with the script
'scripts/decode_stacktrace.sh' in the kernel source tree to get the
names to the symbols of the stack trace.

OpenWRT already provides the CONFIG_COLLECT_KERNEL_DEBUG -option that
creates a kernel with debug-symbols in the OpenWRT output directory.
This change enables this option and copies the generated kernel to the
gluon output directory.

Signed-off-by: Chrissi^ <chris@tinyhost.de>
2020-06-04 22:35:58 +02:00
Matthias Schiffer
7279c4017e
build: target_lib: allow to specify multiple factory_ext values
By passing a table instead of a single string, multiple different
extensions can be specified, each refering to a separate image file
generated by OpenWrt.

This is not supported for sysupgrade (as there can only be a single
image in the format expected by OpenWrt).
2020-05-31 02:20:58 +02:00
Matthias Schiffer
8c386719eb
build: target_lib: do not pass manifest_aliases when defining factory and extra images
manifest_aliases only make sense for sysupgrade images.
2020-05-31 02:20:58 +02:00
Matthias Schiffer
3ce43329f5
build: remove now-unneeded function from target_config_lib.lua
target_config.lua and target_config_check.lua don't pass a table of
callbacks anymore, so target_config_lib.lua can by simplified by moving
all the code that was in the returned function to the toplevel.
2020-05-31 02:20:58 +02:00
Matthias Schiffer
9e23534ec3
build: rework config generation
So far, we were using a sort operation on the generated .config to
implement precedence of =y packages over =m, and =m over unset.
Unfortunately, this sort not only used for packages, but for all config
lines. This made it impossible to override settings from targets/generic
in a target config when the new setting was sorted before the generic
setting.

To fix this, track configurations by their keys, so we can properly
override config keys that were set before. Value-based precedence is
only preserved for package configuration.

The config() and try_config() calls always take key and value as
separate arguments now. Strings are quoted automatically; the values
true, nil and false map to y, m and unset for tristate options. config()
can take an optional third argument to override the error message to
display when the setting fails to apply.

All existing target configs generate the same .config with the old and the
new code. The new code is also a bit faster on targets with many devices.
2020-05-31 02:20:58 +02:00
Matthias Schiffer
14ab51876e
scripts: update-patches: fix shellcheck 0.7.1 lint 2020-05-25 01:10:18 +02:00
Sven Roederer
184dab8fc1 build: ignore deactivated feeds
The OpenWrt feeds.conf.defaults contains some feeds that are commented out
and not active. Such feeds will be returned by the default_feeds.sh script
anyway and causing pseudo feeds. Limit the script to only return active
feeds, by filtering out lines starting with '#'.
This usually only applies to the OpenWrt master branch.

Signed-off-by: Sven Roederer <freifunk@it-solutions.geroedel.de>

change as per NeoRider
2020-05-12 19:46:36 +02:00
Matthias Schiffer
4540217342
build: target_config_check: dedup error messages
Certain error message (for example invalid package names) were emitted
once for each device.
2020-05-11 00:09:13 +02:00
Matthias Schiffer
53690d83be
build: move package list generation to target_config_lib.lua, fix precedence
The precedence of different package lists was broken since #1876,
disallowing removal of GLUON_FEATURES packages via GLUON_SITE_PACKAGES.

Including all package selections, both implicit defaults and explicit
handling in Gluon, the order of precedence is now the following:

1.  OpenWrt defaults (including target-specific defaults)
2.  Device-specific packages from OpenWrt
3.  Generic default packages (from target/generic)
4.  Target default packages (target/$(GLUON_TARGET))
5.  Removal of opkg for tiny targets
6.  Packages derived from GLUON_FEATURES + GLUON_FEATURES_$(class)
7.  GLUON_SITE_PACKAGES
8.  GLUON_SITE_PACKAGES_$(class)
9.  Device-specific packages from target/$(GLUON_TARGET)
10. Device-specific packages from GLUON_$(device)_SITE_PACKAGES

This also contains various pieces of cleanup:

- No hardcoded order of device classes for target_config.lua arguments
  anymore (in fact, the Makefile doesn't know anything about device
  classes now)
- target_conifg_lib.lua only hardcodes the fallback class for x86, no
  other occurences of specific class names
- Feature -> package list mapping is moved from Makefile to the Lua code
  as well (still implemented in Shell though)
2020-05-03 22:05:20 +02:00
Matthias Schiffer
f9062bda0b
build: remove exec prefix from commands with a raw suffix
Allows to append additional commands, for example using `||`.
2020-05-03 18:24:32 +02:00
Matthias Schiffer
89efe55c7c
build: implement image site_packages using a generic site_var function 2020-05-03 18:00:35 +02:00
Martin Weinelt
b181803ac4 scripts: check_site: add need_number_range check 2020-04-30 01:54:35 +02:00
Matthias Schiffer
aefb0b8c35 scripts/copy_output.lua: delete images from OpenWrt dir after copying them
We don't move the images directly, as multiple images of the same device
may have the same source image (on x86), but only delete them after a
whole device has been handled (multiple devices using the same images
must be handled using aliases or manifest aliases instead).
2020-04-25 21:35:44 +02:00
Matthias Schiffer
e993f22233 scripts/target_lib.lua: reorganize images into per-device lists 2020-04-25 21:35:44 +02:00
Matthias Schiffer
584d215f56 build: allow overriding the base feed list 2020-04-09 14:30:44 +02:00
David Bauer
c9f90c3ef8 build: add class-packages for targets without devices
When adding device classes, targets without devices such as x86 were not
handled. As site and feature packages are included on such a per-device
decision, x86 images ended up without most packages.

Include a class setting for a target and include the class-packages
target-wide when this setting is configured.

Fixes 9c52365077 ("build: introduce device classes")
2020-03-27 23:35:24 +01:00
David Bauer
9c52365077 build: introduce device classes
This commit allows to define a device-class flag in the target
definitions. This way, it is possible to distinguish between groups
of devices in the build-process in terms of package or feature
selection.
2020-03-25 02:05:18 +01:00
Matthias Schiffer
d7e724ada9
target_lib: replace envtrue with more intuitive istrue helper
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2020-03-14 16:51:43 +01:00
Jan-Tarek Butt
a2ced263ab
build: add make targets lint, lint-sh, lint-lua
[Matthias Schiffer: minor script improvements]
2019-12-29 15:18:50 +01:00
Jan-Tarek Butt
dd76e0898d
treewide: solve shellcheck warnings 2019-12-14 18:35:11 +01:00
Matthias Schiffer
78054cddbf
scripts: target_config_lib: fix inclusion of device-specific packages from site.mk
Fixes: 071cf7b20f ("Switch to Lua for target definitions")
2019-09-14 13:46:56 +02:00
Christof Schulze
afa3e89890 scripts/update.sh: add verbose error message when commit cannot be found (#1794)
This displays the branch, repo and commitID that was attempted to update to but
failed. Users then can compare this output to the configuration they meant to
activate in modules and hopefully better find their typos.
2019-08-21 23:48:06 +02:00
Matthias Schiffer
09a09d634a
scripts: avoid global variables in target handler scripts 2019-06-18 01:34:45 +02:00
Matthias Schiffer
c9f1fdf6ee
scripts: check_site: use setfenv to keep global environment clean
This is now much nicer to luacheck.
2019-06-17 20:36:35 +02:00
bobcanthelpyou
4249d65af7 treewide: fix luacheck warnings 2019-06-16 22:51:53 +02:00
Matthias Schiffer
912490c026
Add GLUON_DEPRECATED flag
This new build flag is mandatory for now (it may default to 0 in a future
Gluon version). It may be set to the following values:

* 0 - Do not build any images for deprecated devices.
* upgrade - Only build sysupgrade images for deprecated devices.
* full - Build both sysupgrade and factory images for deprecated devices.

"Other" images are handled like factory images, as they are also used for
the initial installation of Gluon on a device.
2019-06-15 23:34:07 +02:00
Matthias Schiffer
071cf7b20f
Switch to Lua for target definitions
The old bash-based parsing code was way too complex. Replace it with Lua.
2019-06-15 23:34:07 +02:00
Matthias Schiffer
16a78b4e9e
scripts/check_site.lua: use new format() function to simplify var_error() 2019-04-28 19:17:47 +02:00
Matthias Schiffer
5abddd2976
scripts/check_site.lua: fix array_to_string with non-string/number array elements
Fixes need_one_of() with boolean elements.
2019-04-28 19:16:28 +02:00
Martin Weinelt
423aafbd29 gluon-core: improve channel and add chanlist validation 2019-04-28 18:17:40 +02:00
bobcanthelpyou
fe521db441 build: add site check for obsolete settings (#1702) 2019-04-28 14:38:08 +02:00
Petr Štetiar
9dc1ac7f58
build: replace hardcoded paths with overridable variables
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-04-22 22:00:45 +02:00
bobcanthelpyou
2c4f022994 build: rename DEVICES to GLUON_DEVICES (#1686) 2019-04-12 01:05:04 +02:00
David Bauer
c1b9ea2d9c gluon-scheduled-domain-switch: add package (#1555)
This package allows to automatically switch to another domain, either
at a given point in time or after the node was offline long enough.
2019-02-12 11:00:29 +01:00
David Bauer
6e48dedeb8 scripts: include binaries in patch
This commit allows for binary content in patches created via 'make
update-patches'.

Binary patches are needed for BDFs required for al boards on the new
ipq40xx target. Before this commit, binaries were omitted from the
generated patch which resulted in the patch not able to apply.

Patches without binary content do not differ in case they are created
with the '--binary' flag.
2018-07-16 01:39:36 +02:00
Matthias Schiffer
7460431faa
scripts/copy_output.sh: add support for "extra" images
The new extra_image command allows to copy additional images that don't
fall into the sysupgrade or factory categories to the output directory.
These images will be placed in the "other" subdirectory.

The extra_image command takes 3 arguments: in_suffix (suffix in OpenWrt
bin directory), out_suffix (suffix in Gluon output directory) and ext (file
extension). Example:

  extra_image '-tftp' '-tftp' '.bin'

The ext argument can be omitted to control the whole filename suffix
including the extension from in_suffix and out_suffix. Multiple extra_image
commands can be used for a single device; aliases work as usual.

Based-on-patch-by: David Bauer <mail@david-bauer.net>
2018-07-14 18:33:37 +02:00
Matthias Schiffer
e8cb34bae2
scripts/copy_output.sh: refactor 2018-07-14 18:17:36 +02:00
Matthias Schiffer
63c16c01a0
scripts: move DEFAULT_FEEDS definition to a separate file
modules.sh can't depend on openwrt/feeds.conf.default, otherwise the
initial update will fail.
2018-07-11 21:47:50 +02:00
Matthias Schiffer
210d97c53e
Switch to OpenWrt 18.06 branch 2018-07-10 22:57:40 +02:00
Matthias Schiffer
a49cf395bd
cripts/check_site.lua: print paths separated by '.' rather than '/'
Using slashs instead of dots is a bit unexpected, and as long as we're not
going for full-blown JSONPath, neither is more correct than the other.
2018-06-29 20:47:59 +02:00
Matthias Schiffer
aeae9df001
check_site: improve error message for unset values 2018-04-27 17:48:36 +02:00
Matthias Schiffer
ad7d154c5a
scripts/check_site.lua: show type of incorrect values in error messages 2018-04-01 23:21:18 +02:00
Matthias Schiffer
b7be32d9f4
scripts/check_site.lua: print proper error message for non-string keys in need_alphanumeric_key() 2018-04-01 23:18:53 +02:00
Martin Weinelt
85c0bd6f57
scripts/check_site.lua: add table_keys() helper 2018-03-20 17:23:48 +01:00
Matthias Schiffer
1a7d93a2b9
scripts/check_site.lua: add support for alternative checks 2018-03-20 17:15:55 +01:00
Matthias Schiffer
6ae067cb37
gluon-core: add domain aliases and pretty name
Based-on-patch-by: lemoer <git@irrelefant.net>
2018-01-26 12:32:47 +01:00