Isolate Podman tests in namespaces (#6499)

* Rename SetProjectName into GetProjectName

Co-authored-by: Anand Singh <ansingh@redhat.com>
Co-authored-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Generate specific containers.conf file for each test spec using a dedicated engine namespace

Co-authored-by: Anand Singh <ansingh@redhat.com>
Co-authored-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Listen on random ports on Podman when '--random-ports' is used

This reduces the risks of port conflicts when running test specs in parallel.

Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Exclude Gosec G404 (use of math/rand) rule

Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Run Podman specs in parallel

* Output the Pod spec to be played by Podman depending on verbosity level

This will help debug potential issues.

* Use random name in 'using devfile that contains K8s resource to run it on podman' test

* Use random component name in sample java-quarkus project used in 'a hotReload capable project is used with odo dev' test

* Revert "Run Podman specs in parallel"

Parallelization works great on GitHub Actions, but I experimented a lot
of issues when running locally with a lot (~11) of parallel test nodes.
Not sure why exactly, but some containers created by Podman had a lot of
networking issues.
We can look into parallelizing the runs later in a subsequent PR.

This reverts commit 64d5d31248a62f355a32ca245ba399a723fdb22f.

* Allow overridding the number of parallel nodes for Podman integration tests

This way, we could be able to run them in parallel on GitHub
but sequentially (default) locally.
This will still benefit us by reducing the time it takes to run such tests on GitHub.

Meanwhile, we can look into the issues we have locally with parallelization.

Note that it is still possible to run them locally in parallel via
the PODMAN_EXEC_NODE env var.

Co-authored-by: Anand Singh <ansingh@redhat.com>
Co-authored-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>
This commit is contained in:
Armel Soro
2023-01-23 15:25:57 +01:00
committed by GitHub
parent f089cba02a
commit 511792a62c
13 changed files with 101 additions and 18 deletions

View File

@@ -47,12 +47,20 @@ UNIT_TEST_ARGS ?=
export ARTIFACT_DIR ?= .
ifdef PODMAN_EXEC_NODES
PODMAN_EXEC_NODES := $(PODMAN_EXEC_NODES)
else
PODMAN_EXEC_NODES := 1
endif
GINKGO_FLAGS_ALL = $(GINKGO_TEST_ARGS) --randomize-all --slow-spec-threshold=$(SLOW_SPEC_THRESHOLD) -timeout $(TIMEOUT) --no-color
# Flags to run one test per core.
GINKGO_FLAGS_AUTO = $(GINKGO_FLAGS_ALL) -p
# Flags for tests that may be run in parallel
GINKGO_FLAGS=$(GINKGO_FLAGS_ALL) -nodes=$(TEST_EXEC_NODES)
# Flags for Podman tests that may be run in parallel
GINKGO_FLAGS_PODMAN=$(GINKGO_FLAGS_ALL) -nodes=$(PODMAN_EXEC_NODES)
# Flags for tests that must not be run in parallel
GINKGO_FLAGS_ONE=$(GINKGO_FLAGS_ALL) -nodes=1
# GolangCi version for unit-validate test
@@ -202,9 +210,12 @@ test-integration-openshift-unauth:
test-integration-no-cluster:
$(RUN_GINKGO) $(GINKGO_FLAGS_AUTO) --junit-report="test-integration-nc.xml" --label-filter=nocluster tests/integration
# Running by default on 1 node because of issues when running too many Podman containers locally,
# but you can override this behavior by setting the PODMAN_EXEC_NODES env var if needed.
# For example: "PODMAN_EXEC_NODES=5".
.PHONY: test-integration-podman
test-integration-podman:
$(RUN_GINKGO) $(GINKGO_FLAGS_ONE) --junit-report="test-integration-podman.xml" --label-filter=podman tests/integration
$(RUN_GINKGO) $(GINKGO_FLAGS_PODMAN) --junit-report="test-integration-podman.xml" --label-filter=podman tests/integration
.PHONY: test-integration
test-integration: test-integration-no-cluster test-integration-cluster