Commit Graph

76 Commits

Author SHA1 Message Date
lemoer
c133fbbef6
push_pkg.sh: use scp protocol for openssh scp (#2479)
Starting from OpenSSH 9.0p, scp started to use sftp in favor of the
scp protocol by default. As dropbear in OpenWrt currently does not
support sftp by default, we now use the fallback cli switch "-O"
to use the scp protocol for scp.
2022-05-28 20:41:10 +02:00
lemoer
5f8da70ffd docs: adjust example site.conf 2022-05-21 20:08:52 +02:00
Matthias Schiffer
fc21c51ab2
ci: build all targets on changes in scripts/
The scripts directory contains most of our build system, so changing it
should trigger a build of all targets.
2022-03-23 23:40:24 +01:00
Matthias Schiffer
db80146093
Merge pull request #2368 from freifunk-gluon/lint-sh-fixes
lint-sh fixes
2022-02-03 16:37:18 +01:00
Matthias Schiffer
c7e38fdc2c
contrib: lsupgrade: optimize output
- Use printf instead of echo for better portability
- Print whole path without reordering components
- Deduplicate code
2022-01-23 14:44:06 +01:00
Matthias Schiffer
c0afb9cc39
contrib: lsupgrade: do not search the OpenWrt "packages" feed
Iterating over all the package directories in the OpenWrt feed takes a
while, even though it doesn't contain any upgrade scripts. Skip the
whole directory.
2022-01-23 14:44:06 +01:00
Matthias Schiffer
24682a3197
contrib: push_pkg: fix lint issues with shellcheck 0.7.1
0.7.1 complains about unescaped backslashes in double quotes (which are
intepreted by printf in the two affected places). While the warning was
retired with shellcheck 0.7.2, it seems like a good idea to fix it
anyways.
2022-01-03 03:25:27 +01:00
Martin Weinelt
6728c4a103
Add helper to run a containerized build environment (#2292)
Using `make container` or, if you don't have automake/gmake on your host
system, `./scripts/container.sh` will build an image for the current
branch your are on and drop you into a shell running inside a container
using that image.

From there all tooling required to work on Gluon is available.

Supports both podman (preferred) and docker.
2021-12-21 21:40:27 +01:00
lemoer
27cd003d1b actions: rebuild targets if includes change 2021-09-19 21:24:17 +02:00
Martin Weinelt
b4677b8b88 contrib/ci/minimal-site: build with owe and wpa3 support 2021-09-06 02:37:45 +02:00
Martin Weinelt
e4086446cc
Use python3 in CI and documented build requirements
This has become possible with OpenWrt 21.02 and while the Dockerfile
already received that updated, we forgot to update the CI and the
documentation.
2021-08-12 01:27:50 +02:00
Martin Weinelt
3bb3f02bcd contrib: drop Jenkins setup
Has gone unused for a while now and if it is needed again it can be
retrieved from the history.
2021-08-10 22:42:19 +02:00
Martin Weinelt
1f8c5d1c8f
actions: build targets conditionally 2021-08-10 22:16:08 +02:00
lemoer
d4dee692c6
contrib: add push_pkg.sh to enhance package development workflow (v2) (#2023)
v2: In contrast to the last patches, this is now built on top of ssh
    only, without using e.g. 9pfs. Furthermore it works also with
    arbitary remote hosts on any target/architecture. Also the
    scripts were renamed and moved to /scripts.

The aim of this commit is to allow fast rebuild cycles during the
development of gluon packages.

Currently the following workflow can be used:

	# start a local qemu instance
	scripts/run_qemu.sh output/images/factory/[...].img

	# do your changes in the file you want to patch
	vi package/gluon-ebtables/files/etc/init.d/gluon-ebtables

	# rebuild and update the package
	scripts/push_pkg.sh package/gluon-ebtables/

	# test your changes
	...

	# do more changes
	...

	# rebuild and update the package
	scripts/push_pkg.sh package/gluon-ebtables/

	# test your changes
	...

	(and so on...)

Implementation details:
- Currently this is based on ssh/scp.
- Opkg is used to install/update the packages in the remote machine.

Benefits:
- This works with compiled and non-compiled packages.
- This works with native OpenWrt and Gluon packages.
- This even performs the check_site.lua checks as they are integrated
  as post_install scripts into the openwrt package.
- It works for all architectures/targets.
2021-07-23 19:50:15 +02:00
David Bauer
2aa6f90bb3 docs contrib: add dependency on qemu-utils 2021-06-22 01:59:48 +02:00
Martin Weinelt
f4b0d007f3 contrib: Dockerfile: install rsync
The kernel build requires rsync.
2021-06-22 01:59:48 +02:00
Martin Weinelt
4d0b8ccca3 contrib: Dockerfile: add python3, drop python2 2021-06-22 01:59:48 +02:00
David Bauer
53b647f748 actions: build gluon for next sub-branches
Currently we do not perform CI firmware builds on the next-2102 branch.

Build Gluon for all branches starting with "next" to increase the
coverage of our build tests.

Signed-off-by: David Bauer <mail@david-bauer.net>
2021-05-01 13:35:36 +02:00
Vieno Hakkerinen
09d6f70b07
contrib/Dockerfile: use apt-get instead of apt
* apt is meant for user interactive usage. apt does not guarantee a stable CLI.
* set DEBIAN_FRONTEND=noninteractive to tell apt-get that no user interaction is wanted
2021-04-07 18:01:56 +02:00
Vieno Hakkerinen
ca7a8ff528
contrib/actions: use apt-get instead of apt
apt does not have a stable CLI interface. Don't use it in scripts.
2021-04-05 21:33:04 +02:00
Martin Weinelt
394bc8e6bd
contrib/actions: drop custom sources.list 2021-04-05 04:11:29 +02:00
Martin Weinelt
e6b996f0e1 Revert "actions: pin Ubuntu version"
This reverts commit d9621048ed.
2021-04-05 04:09:42 +02:00
David Bauer
d9621048ed actions: pin Ubuntu version
ubuntu-latest is now assigned to Ubuntu 20.04. As we use custom apt
sources for 18.04, pin to this version for now to fix the CI.
2021-02-15 01:38:51 +01:00
Martin Weinelt
bab05c8515
actions: run tasks based on set of modified paths 2021-01-02 17:25:51 +01:00
David Bauer
441fbf1283 actions: update generation script to use matrix 2021-01-02 15:06:16 +01:00
Tim Gates
6b9e3ce1df
docs: ci: fix simple typo (#2162) 2020-12-19 20:50:54 +01:00
David Bauer
0b1ee0cf69
github: remove GitHub actions cache (#2120)
The performance benefit the cache brought was due to a broken sources
CDN mirror handling in OpenWrt.

The cache brings no measurable performance benefit. Disable it to slim
down the pipeline steps.
2020-09-18 13:50:02 +02:00
David Bauer
de9d3a917e
actions: cache OpenWrt DL folder (#2084)
This adds the ability to cache OpenWrt dependencies on a per-target
base. Artifacts over 10MB are excluded, as GitHub imposes a limit of 5G
of available space per repository cache. This affects mostly
linux(-firmware) and gcc / gdb.

The goal is to reduce the total amount of requests necessary to fetch
dependencies.
2020-07-25 00:42:55 +02:00
Matthias Schiffer
6b3b5e706e jenkins-community-slave: use requests library to download slave.jar
Do not mix two different HTTP libraries in one script. This fixes a
warning:

/var/lib/jenkins/slave.py:82: DeprecationWarning: URLopener style of invoking requests is deprecated. Use newer urlopen functions/methods

This simple implementation has the disadvantage that it stores the whole
file in RAM, but this isn't an issue for the 1.5MB slave.jar.
2020-06-01 20:18:24 +02:00
Matthias Schiffer
5232428b19 jenkins-community-slave: verify HTTPS certificate in head() request
Fixes a warning:

/usr/local/lib/python3.7/dist-packages/urllib3/connectionpool.py:1004: InsecureRequestWarning: Unverified HTTPS request is being made to host 'build.ffh.zone'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
2020-06-01 20:18:24 +02:00
Martin Weinelt
7fdffcaa1c
Jenkinsfile: require gluon-vmx label for qemu testing
This allows us to make use of builders that can't run virtual machines.

Drop the TMUX environemnt variable, it's not needed anymore.
2020-06-01 19:54:22 +02:00
Matthias Schiffer
55c51f2a73 actions: script improvements
- Rely on shebang instead of setting shell in workflow
- Run whole install-dependencies.sh in sudo
- Use /bin/sh instead of bash
- set -e
2020-05-13 15:48:06 +02:00
Matthias Schiffer
2889904e5f actions: set BUILD_LOG=1 and store logs as artifacts
The new step is added with `if: ${{ !cancelled() }}`, so the logs are
stored even when the build fails.
2020-05-13 15:48:06 +02:00
Matthias Schiffer
ab6596115a build: introduce GLUON_AUTOREMOVE flag 2020-05-13 15:48:06 +02:00
David Bauer
0d341a76b3 actions: create per-target artifacts 2020-05-07 02:29:11 +02:00
Martin Weinelt
dd7d37b8e4 Jenkinsfile: add timeout to build and test steps
This will hopefully make CI runs more robust and recover stale workers.
2020-04-26 23:26:09 +02:00
Martin Weinelt
42650393b6
contrib/ci: document kvm permissions configuration 2020-04-26 00:25:11 +02:00
Martin Weinelt
5c8c479d21 Jenkinsfile: fix glob for test scripts 2020-04-17 20:00:57 +02:00
Martin Weinelt
777416a175
Jenkinsfile: require gluon-docker-v2 for build stage
This is required to build the next branch which requires rsync to be
available.
2020-04-15 23:05:57 +02:00
Martin Weinelt
358be325b2
contrib/ci: install rsync on jenkins-community-slaves
Rsync is a requirement for OpenWrt master and therefore our next
branch. Currently builds on x86-64 error out due to missing rsync:

/bin/sh: 1: rsync: not found
2020-04-15 17:56:38 +02:00
Martin Weinelt
d9ea281436 ci: identify runner in each stage 2020-04-15 15:20:45 +02:00
Martin Weinelt
772db95244 contrib/ci: add test stage 2020-04-15 15:20:45 +02:00
David Bauer
d3f5adcfa4 github: actions: only build certain branches
This enables build-testing only on master as well as next and release
branches.
2020-04-10 03:50:31 +02:00
Martin Weinelt
8aa30b7099 ci: add chown step to resolve permissions on openwrt dl cache 2020-04-07 19:26:26 +02:00
David Bauer
1b0288d539 contrib: actions: remove surplus package sources 2020-04-05 19:49:44 +02:00
David Bauer
366ba8ab99 contrib: actions: add custom sources.list 2020-04-05 19:43:57 +02:00
David Bauer
83b2c41cee contrib: actions: enable verbose output 2020-04-05 19:19:58 +02:00
David Bauer
bb905b3351 actions: move dependencies to dedicated step 2020-04-05 19:08:15 +02:00
David Bauer
ebac7e4459 contrib: add build-script for GitHub actions 2020-04-04 18:04:31 +02:00
David Bauer
a37098da6b contrib: add generation script for GitHub actions 2020-04-04 18:04:31 +02:00