Commit Graph

1779 Commits

Author SHA1 Message Date
Parthvi Vala
039ef07320 Bump version to 3.10.0 (#6783)
Signed-off-by: Parthvi Vala <pvala@redhat.com>
2023-05-02 10:04:54 -04:00
Parthvi Vala
010e9634d3 Use custom address for port forwarding (#6766)
* Custom address for port forwarding

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add integration tests

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use private variables for custom address and ports

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Update pkg/util/util.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Assign custom address to HostIP for podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add validation for free port when --port-forward is provided in the Validate() method, add integration test for running parallel dev sessions on same platform, same port and different addresses

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix unit test failure

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Attempt at fixing windows failure

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add documentation

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use default value for --address flag

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Changes from review

Co-authored-by: Armel Soro <asoro@redhat.com>
Signed-off-by: Parthvi Vala <pvala@redhat.com>

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
Co-authored-by: Armel Soro <asoro@redhat.com>
2023-05-02 07:34:19 -04:00
Philippe Martin
191ee6f45f Wait when Build command fails (#6771)
* Same signature for reconcile in kubedev/podmandev

* Do not call initial reconcile

* Do not retry + filter deployment events

* Integration tests

* Move logs back to level 4

* Remove unnecessary files

* Log state + Set state to ready when build fails
2023-05-02 04:39:58 -04:00
Armel Soro
00918c3d16 Support variable substitution with --var and --var-file in odo build-images (#6776)
* Add integration tests

* Add support for --var and --var-file to the 'build-images' command

* Update 'build-images' command reference
2023-04-27 11:13:16 -04:00
Philippe Martin
e9c5f86563 Simplify devfile Kubernetes adapter (#6762)
* Get values from context

* Move Devfile param to WatchParams and biuld adapter only once

* Move pkg/devfile/adapters/kubernetes/* into pkg/dev/kubedev

* Rename Push to reconcile and split in 2 parts: components and innreloop

* Pass out ans errout as startOptions

* Embed StartOptions into PushParameters

* Embed StartOptions into WatchParameters

* Fix passing startoptions

* Deduplicate options (out, ...)

* Revert adding unwanted files

* Fix wait app ready
2023-04-25 05:13:19 -04:00
Philippe Martin
4d1e402542 Check files to sync with gitignore library (#6760) 2023-04-24 16:29:41 +00:00
Philippe Martin
61c38c4205 Add annotation for setting access mode on automounted configmap/secret files (#6750)
* Support accessmode annotation

* Support accessmode annotation

* Add ddoc

* Add test for decimal value + more doc
2023-04-22 16:31:27 +00:00
Armel Soro
4bab9285fb Allow to cancel execution of odo dev at any phase (e.g. if build command is taking long) (#6736)
* Add test cases simulating build or run commands that take very long

* Pass a context around to relevant functions and methods

This will allow to handle cancellations and timeouts and deadlines as needed

* Pass the context to Podman exec command too

* fixup! Add test cases simulating build or run commands that take very long
2023-04-20 17:12:28 +00:00
Philippe Martin
b843ce845e Set vcs-uri annotation on podman (#6746)
* Set vcs-uri annotation on podman

* Integration test

* Disable test with Podman v3

* Make test run with podman v4

* Fix displaying error
2023-04-19 11:35:18 -04:00
Philippe Martin
bd5098eb53 Use devstats.PID.json as devstate (#6713)
* Use devstate.PID.json

* odo describe component gets forwarded ports from devstate.json

* Pass pid in context

* Add platform to state

* Overwrite devstate.json if PId not exists

* odo describe component displays forwarded ports from podman/cluster

* Start only one session on platform

* Fix integration test

* Review

* Fix

* Update pkg/state/errors.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Integration tests

* Fix integration test

---------

Co-authored-by: Armel Soro <armel@rm3l.org>
2023-04-19 04:16:08 -04:00
Parthvi Vala
912f6d14e6 Fix flaky unit Test_getCompleteCustomPortPairs (#6737)
* Fix flaky unit Test_getCompleteCustomPortPairs

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use range check

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Revert "Use range check"

This reverts commit 1bcbacf184.

* Use order to iterate over ceMapping

Signed-off-by: Parthvi Vala <pvala@redhat.com>

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
2023-04-18 10:18:49 -04:00
Parthvi Vala
28ba3df556 Podman custom port mapping (#6729)
* Temp Change

* Custom port mapping works with podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add unit tests

* Move custom port forward formation to the general getPortMapping function

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix unit tests

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Refactor integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

Add dev --debug integration test for podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>

Add dev --debug integration test for podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>

Fix ci failures

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use order to iterate over ceMapping and fix a check with debug integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
2023-04-18 10:01:49 -04:00
Philippe Martin
7a721348b3 Automount volumes (#6698)
* Simplify AddOdoProjectVolume and AddOdoMandatoryVolume

* Rename / Clarify HandleEphemeralStorage function

* Regroup volume-specific code

* Move volume specific code to a separated function

* Add a new module configAutomount

* Automount PVC (without options)

* Add unit tests

* Separate functions

* Mount secrets

* Mount configmaps

* Specific mount path

* MountAs annotation

* Mounting cm/secret as env

* Refacto: use inAllContainers + replace result with volume

* Mounting cm/secret as subpath

* Read-only

* Integration tests

* Rename label

* Automount during odo deploy Exec command

* Add documentation

* Fix TODO

* Review

* Fix indentation

* Rename labels/annotations
2023-04-18 07:58:45 -04:00
Parthvi Vala
7f25f0cf61 Fix: PodSecurity Admission: Display duplicate warnings only once (#6733)
Signed-off-by: Parthvi Vala <pvala@redhat.com>
2023-04-14 01:15:57 -04:00
Armel Soro
72ea3cd9e5 Wait until expected ports are opened in the container before starting port-forwarding (#6701)
* Add function in 'port' package allowing to check whether given ports are actually opened and in LISTEN state in specified containers

* Wait on Kubernetes until the application is ready by checking the ports to forward to are actually opened (or a timeout expires)

This allows to display port-forwarding messages when the application is ready to accept requests.
Otherwise, if the application takes time to listen on the port,
and we try to reach the local forwarded port, port forwarding will
be restarted.
If we are using --random-ports, new random ports will be generated.

Known issue: with the backo-go exponential backoff implementation,
it seems hitting Ctrl-C does not stop waiting =>
we need to wait until the timeout is reached or the backoff is done.

* Wait on Podman until the application is ready by checking the ports to forward to are actually opened (or a timeout expires)

This allows to display port-forwarding messages when the application is ready to accept requests.
Otherwise, if the application takes time to listen on the port,
and we try to reach the local forwarded port, port forwarding will
be restarted.
If we are using --random-ports, new random ports will be generated.

Known issue: with the backo-go exponential backoff implementation,
it seems hitting Ctrl-C does not stop waiting =>
we need to wait until the timeout is reached or the backoff is done.

* Display warnings instead if the ports are not opened

While iterating on the application, it might feel
weird to stop the Dev Session immediately.
Plus, a lot of integration tests are not passing because
of source code not strictly matching the ports declared in the
(too many) Devfiles.

* Fix integration test checking the behavior when simulating an app taking long to start

* Fix output expected for documentation tests

* Add hint to run 'odo logs --follow [--platform]' to help understand why app is not listening on the expected ports

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* Collect all unreachable ports along with the containers and return the list to the users

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* Apply suggestions from code review

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* fixup! Add hint to run 'odo logs --follow [--platform]' to help understand why app is not listening on the expected ports

* fixup! Add hint to run 'odo logs --follow [--platform]' to help understand why app is not listening on the expected ports

---------

Co-authored-by: Parthvi Vala <pvala@redhat.com>
2023-04-13 11:56:22 -04:00
Parthvi Vala
e6a7f0253b Fix: random port matches a custom local port (#6727)
Signed-off-by: Parthvi Vala <pvala@redhat.com>
2023-04-12 02:42:32 -04:00
Armel Soro
c2d850fa41 Follow-up to comments on #6654 (support for autoBuild and deployByDefault) (#6720)
* Remove additional non-nil checks on component fields since they are already filtered by type

We are confident that the fields we expect
(comp.{Image,Kubernetes,OpenShift}) are non-nil.

* Rename 'libdevfile.GetImageComponentsToPush' into 'libdevfile.GetImageComponentsToPushAutomatically'

* Document with examples cases where 'deployByDefault' and 'autoBuild' are `false` and the component is not referenced

* Rename 'image-autobuild-true-and-referenced' command into 'image-autobuild-true' in test Devfile

* Simplify parsing the output on Podman in the tests when looking for K8s/OpenShift components

* Use 'source/nodejs' instead of 'source/devfiles/nodejs/project' in the tests

* Revert "Rename 'image-autobuild-true-and-referenced' command into 'image-autobuild-true' in test Devfile"

This reverts commit 65812b857c.
2023-04-07 03:07:43 -04:00
Parthvi Vala
c82583bdc3 Allow users to define custom port-mappings for port forwarding [Kubernetes] (#6704)
* Allow users to define custom port-mappings for port forwarding: add methods to parse and validate  the flag

* Add methods to kube port-forwarding to obtain custom port pairs

Co-authored-by: Armel Soro <asoro@redhat.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Pass custom port-mappings from CLI to Business layer

Co-authored-by: Armel Soro <asoro@redhat.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Add integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add integration test for debug

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add check for using --random-ports and --port-forward together

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Return all validation errors for --port-forward flag at once

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use random local ports for integration testing

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix ci failures

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Print invalids of --port-forward to stderr so that they are printed when debugging

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Attempt at fixing the test failure on windows

Signed-off-by: Parthvi Vala <pvala@redhat.com>

Remove the use of seed for randomzing

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use global counters for generate port number

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Apply changes from code review

Co-authored-by: Armel Soro <asoro@redhat.com>

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <asoro@redhat.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>
2023-04-06 15:37:36 -04:00
Armel Soro
8cb53c2a04 Bump version to 3.9.0 (#6717) 2023-04-05 05:06:19 -04:00
Armel Soro
cc2e8def81 Support autoBuild and deployByDefault on Image and Kubernetes/OpenShift components (#6654)
* Add sample Devfile with multiple autoBuild/deployByDefault combinations

It will be used for integration tests.

* Add helper function to update a given Devfile Command Group

This will allow supporting cases where we need to run a custom command,
but this is not implemented yet on odo (cases with 'odo dev --debug'
and 'odo deploy').
In this case, this helper will allow updating the Devfile for example to
unmark the current default command as non-default, and mark the custom
one as default.

* Add test cases for 'odo dev'

* Add test cases for 'odo deploy'

* Add test cases for 'odo build-images'

'odo build-images' should build all images regardless of the 'autoBuild' property.

* Display the spinner when creating or updating Kubernetes resources

This helps understand what resources are being patched.

* Handle deployByDefault on K8s and OpenShift components

* Add 'devfile.GetImageComponentsToPush' functions that returns the list of image components to auto-create

See https://github.com/devfile/api/issues/852#issuecomment-1211928487 for more context.

* Handle automatic image component creation for 'odo dev' on Kubernetes

* Handle automatic image component creation for 'odo dev' on Podman

* Handle automatic image and K8s/OpenShift component creation for 'odo deploy'

* Bump Devfile library to the latest commit at this time (c1b23d2)

This includes the fix for [1], which provides a way not to set default values automatically

[1] https://github.com/devfile/api/issues/1067

* Do not set default values when parsing a Devfile

See [1] for the rationale

[1] https://github.com/redhat-developer/odo/issues/5694\#issuecomment-1465778398

* Add documentation in the Devfile reference page

* Revert "Display the spinner when creating or updating Kubernetes resources"

This reverts commit 6ad073e63cb0e685f165eed767619a90997393a3.

* Avoid re-applying Image components multiple times

'adapter#Push' might get called several times when there are
state transitions (either on the Deployment in the cluster,
or from 'odo').
It might be confusing to apply Image components over and over again
(and also it can be slower if we need to push images to remote registries).

* Move GetK8sAndOcComponentsToPush and GetImageComponentsToPush to libdevfile package

As suggested in review, this should be the responsibility of the devfile library

Co-authored-by: Philippe Martin <phmartin@redhat.com>

* fixup! Handle automatic image and K8s/OpenShift component creation for 'odo deploy'

* fixup! Handle automatic image component creation for 'odo dev' on Podman

* fixup! Avoid re-applying Image components multiple times

* Apply suggestions from code review

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* fixup! Do not set default values when parsing a Devfile

* Fix devfile-deploy-functional-pods.yaml (used in 'odo logs' tests)

Set deployByDefault to false on innerloop-pod component.
Otherwise, since it is not referenced by any apply command,
it will be automatically created by *both* 'odo dev' and 'odo deploy'.

---------

Co-authored-by: Philippe Martin <phmartin@redhat.com>
Co-authored-by: Parthvi Vala <pvala@redhat.com>
2023-04-05 01:30:30 -04:00
Philippe Martin
252a77dde7 Speedup odo registry --details (#6678)
* Do not use deprecated DefaultStarterProjects

* Fix integration tests

* Use CommandGroups from registry index

* Fix JSON field name / Detailed err message

* Clarify --details is to be used with --devfile

* Document versions.commandGroups in json output
2023-04-03 10:05:04 -04:00
Armel Soro
25bb53ff30 Fix issue preventing app from being restarted if /proc/[pid]/task/[tid]/children is missing in container (case of Minikube with VirtualBox or KVM drivers) (#6690)
* Get the process children by reading the "/proc/*/stat" files instead of relying on "/proc/*/task/*/children"

Co-authored-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Fix unit tests + encapsulate into getProcessChildren

* Make getProcessChildrenRec a closure inside k.getProcessChildren

* Rename kubexec_test.go into kubeexec_test.go to respect naming convention on test files

* fixup! Make getProcessChildrenRec a closure inside k.getProcessChildren

* Update pkg/remotecmd/kubeexec.go

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* fixup! Update pkg/remotecmd/kubeexec.go

* Remove TODO note about handling missing children file

The implementation no longer relies on the presence of this file.

* Make getProcessChildren return only children PIDs, without the root ppid

This makes the intent of this method clearer, so we don't try to kill the root ppid twice.

* Fix test method names in 'kubeexec_test.go' file

---------

Co-authored-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>
2023-04-03 07:00:46 -04:00
Parthvi Vala
dfb36ddc56 odo dev/deploy should display a warning about default namespace on cluster (#6688)
* odo dev/deploy should display a warning about default namespace on cluster

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Move warning to after odo title

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add integration test for odo dev/deploy when not connected to a cluster or podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix podman test failure-s

* Add a new line before warning, and use corev1.NamespaceDefault instead of default

Signed-off-by: Parthvi Vala <pvala@redhat.com>

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
2023-03-29 09:36:15 +02:00
Parthvi Vala
61d53dc99c Add name detection to odo analyze (#6682)
* Add name detection to odo analyze

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Update tests/helper/helper_generic.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update tests/integration/cmd_analyze_test.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update odo analyze documentation

Signed-off-by: Parthvi Vala <pvala@redhat.com>

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
2023-03-28 16:57:29 +02:00
Philippe Martin
163c164d18 Pod Security Admission support for odo deploy (#6679)
* Use GetPodTemplateSpec instead of deprecated GetContainers

* Specify pod admission policy when building Exec Job

* Integration test

* Update tests/helper/component_podman.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* add test

---------

Co-authored-by: Armel Soro <armel@rm3l.org>
2023-03-28 16:23:11 +02:00
Parthvi Vala
c70aaff642 Fix: odo init does not ask for starter project if the Devfile stack contains extra files (#6684)
Signed-off-by: Parthvi Vala <pvala@redhat.com>
2023-03-27 11:18:17 -04:00
Armel Soro
2ef5316bdd Fix UX issue when exec command fails to run as part of a deploy command (#6673)
For the context, before the changes, the "Execution output" message
was displayed on the same line as the spinner, e.g.:

```
↪ Executing command:
◓  Executing command in container (command: exec-deploy)Execution output:
/bin/sh: line 1: helm: command not found
 ✗  Executing command in container (command: exec-deploy) [8s]
 ✗  failed to execute (command: exec-deploy)
```

This is now fixed by ending the spinner as soon as possible,
and then displaying the command output:

```
↪ Executing command:
 ✗  Executing command in container (command: exec-deploy) [8s]
Execution output:
/bin/sh: line 1: helm: command not found
 ✗  failed to execute (command: exec-deploy)

```
2023-03-23 05:58:24 -04:00
Parthvi Vala
47234b94b3 Fix: Starterproject not replace existing devfile stack files of the context dir (#6633)
* Starterproject does not replace existing files of the context dir

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix issue with os.RemoveAll

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix errors, it works now

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix test failures

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use filesystem to perform file operations wherever possible

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use filesystem to perform file operations wherever possible (2)

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add unit test for DownloadStarterProject 1/n

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix zip files

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix zip files (2)

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add unit test starter project with Devfile

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix zip files (3)

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Finish unit tests

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Remove unnecessary integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add comments and verbosity, cleanup code

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Update pkg/registry/registry.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update pkg/util/util.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update pkg/registry/registry.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* rm3l's suggestion on using err when not shadowed and enhance error messages

Signed-off-by: Parthvi Vala <pvala@redhat.com>

Fix validation errors

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Change arg position in IsValidProjectDir

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* rename isConflicting to getConflictingFiles

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add zip files for starterproject with empty dir

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add unit tests covering starterprojects with empty dir for both conflicting and non-conflicting cases

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Handle error while walking

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Delete temp dirs

Signed-off-by: Parthvi Vala <pvala@redhat.com>

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
2023-03-21 08:55:42 -04:00
Philippe Martin
93d1330182 Patch Pods to pass pod security admission (#6602)
* Bump devfile/library

* Use generator.GetPodTemplateSpec instead of Get[Init]Containers

* Fix deprecated pointer.*

* Fix more deprecated

* Workaround for ServerPreferredResources panicking

* Pass policy to GetPodTemplateSpec

* Integration test

* document behaviour

* Review
2023-03-20 10:16:14 -04:00
Armel Soro
86b271d65c Fix issue with conflicting ports when starting multiple Dev sessions on Podman (#6660)
Due to regression on 6.1+ kernels [1], binding on 127.0.0.1:$port after some process
listens on 0.0.0.0:$port does pass, which is not expected.
Until this issue is fixed [2], a possible workaround is to try to bind on 0.0.0.0:$port.
This works correctly on Podman, and also on Kubernetes
(because we do port-forwarding on Kubernetes on 127.0.0.1:$port).

The unit test added should allow us to detect similar issues faster in the future,
should it happen again.

[1] https://lore.kernel.org/stable/e21bf153-80b0-9ec0-15ba-e04a4ad42c34@redhat.com/T/
[2] https://lore.kernel.org/netdev/20230312031904.4674-3-kuniyu@amazon.com/T/
2023-03-20 04:44:54 -04:00
Armel Soro
bffb9b0f35 Fix error handling when trying to delete remote resources not present in Devfile (#6659)
* Fix condition for checking errors while trying to delete resources not present in Devfile

* Add unit tests

* Fix race condition with error handling when deleting remote resources not present in Devfile

Several Goroutines could potentially modify the same shared error variable,
leading to weird behaviour, caught by the unit tests.
Using errgroup allows simplifying goroutine error handling.

* Remove the user-facing spinner while trying to remove remote resources not present in Devfile

Replace with a klog message, as the spinner does not add much value to the end user.

Plus, it keeps displaying because we are regenerating the adapter which calls this function indefinitely.
2023-03-16 10:46:19 -04:00
Philippe Martin
9c592c4f04 Bump to v3.8.0 (#6634) 2023-03-08 08:49:27 +01:00
Armel Soro
f001676d93 Ignore errors that might happen when getting Devfile Registries from the cluster (#6639) 2023-03-07 14:06:57 +01:00
Philippe Martin
26c90d7c4d Remove podman from experimental mode (#6628)
* Remove podman from experimental mode

* Remove env var for odo dev

* Cleanup tests and sources about experimental mode

* Hide "Experimental mode" doc page

* Cleanup latest commit

* doc odo dev --platform podman

* Doc other commands

* Apply suggestions from code review

Co-authored-by: Armel Soro <armel@rm3l.org>

* Remove more ODO_EXPERIMENTAL_MODE from latest commits on main

* Remove reference to Experimental mode from blog posts

---------

Co-authored-by: Armel Soro <armel@rm3l.org>
2023-03-06 14:17:39 +01:00
Armel Soro
4fb7308cfa Automatically use in-cluster Devfile registries if any (#6622)
* Add kubeclient as dependency of the registry client

Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Add GetRegistryList method to the kube client interface

This is yet to be implemented.

Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Implement GetRegistryList

* adding test if devfileRegistryListCR is present in cluster

Signed-off-by: anandrkskd <anandrkskd@gmail.com>

* Unit tests (to be continued)

* Add unit test cases against kclient#GetRegistryList()

Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Ignore in-cluster registries with an empty URL

This should ideally not happen if the registry operator
 is installed in the cluster (because it validates the
 URL to make sure it is reachable), but you never know ;-)

* Update error message when trying to remove registry

Registries might be found in the cluster.

* Pass isSecure value to the registry handler

* Make it possible to use in-cluster registries when calling 'odo registry --details'

* Remove unused 'preferenceClient' from registry.getRegistryStacks

* Handle in-cluster registries in 'odo init' non-interactive mode

* Handle in-cluster registries in 'odo init' interactive mode

* Add integration test for odo init --devfile-registry

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use proxy when available

Signed-off-by: Parthvi Vala <pvala@redhat.com>

Co-authored-by: Armel Soro <armel@rm3l.org>

* Make sure tests work even if the registry operator is installed in the cluster or if there are cluster-wide registry lists

* Add tests for 'odo init' interactive mode

* Remove useless CR file

CRs are now dynamically created and applied from the tests

* fixup! Add tests for 'odo init' interactive mode

---------

Signed-off-by: anandrkskd <anandrkskd@gmail.com>
Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>
Co-authored-by: anandrkskd <anandrkskd@gmail.com>
Co-authored-by: Parthvi Vala <pvala@redhat.com>
2023-03-06 09:19:02 +01:00
Armel Soro
29b5a38f88 Make port-forwarding work on Podman with apps listening on the loopback interface, via a new --forward-localhost flag (#6629)
* Embed platform.Client interface in platform-specific interfaces

This avoids repeating the same methods in both interfaces,
and makes the intent clearer.

* Verify interface compliance of PodmanCli at compile time

This is recommended in the Coding Conventions guidelines [1].
Specifically, what's important here is checking that it meets the 'platform.Client' contract.

[1] https://github.com/redhat-developer/odo/wiki/Dev:-Coding-Conventions#verify-interface-compliance

* Move K8s-specific implementation of port-forwarding to a dedicated package

This paves the way to providing a different implementation for Podman

* Remove GetPortsToForward method from the portForward.Client interface

Current implementation relies on the Devfile object,
so it makes more sense to be in the libdevfile package.

* Monitor and send appropriate status events after starting a remote command process

This allows callers to get more meaningful events about the process.

* Implement port-forwarding logic on Podman

As explained in [1], this makes use of a helper sidecar container
(aptly named "odo-helper-port-forwarding") to be added to the Pod Spec created by odo.
In this scope, port-forwarding will be equivalent of executing a socat command in this
helper container, like so:

socat -d tcp-listen:20002,reuseaddr,fork tcp:localhost:5858

In the command above, this will open up port 20001 on the helper container,
and forwarding requests to localhost:5858
(which would be in the application container, part of the same Pod)

[1] https://github.com/redhat-developer/odo/issues/6510

* Update portForward.Client interface methods

Specifically, the 'StartPortForwarding' method
can now accept an explicit list of ports that needs to
be forwarded, if the caller can compute provide such information.

This is currently useful on Podman where the ports
(even the random ones) are known in advance.

* Add helper sidecar container to the Pod Spec generated on Podman

As explained in [1], this helper sidecar container (aptly named "odo-helper-port-forwarding")
will hold the actual container/host ports mapping in the spec
(to overcome the limitation of using hostPort against a container app listening on the loopback interface);
this running container will be used to execute the actual port-forwarding commands (based on socat), e.g:

```
apiVersion: v1
kind: Pod
metadata:
  name: my-component-app
  labels:
    app: my-component-app
spec:
  containers:
  - name: runtime
    command: [ 'tail', '-f', '/dev/null']
    # Omitted for brevity, but imagine an application being executed by odo
    # and listening on both 0.0.0.0:3000 and 127.0.0.1:5858

  - name: odo-helper-port-forwarding
    image: quay.io/devfile/base-developer-image:ubi8-latest
    command: [ 'tail', '-f', '/dev/null']
    ports:
    - containerPort: 20001
      # A command will be executed by odo, e.g.: 'socat -d -d tcp-listen:20001,reuseaddr,fork tcp:localhost:3000'
      hostPort: 20001
    - containerPort: 20002
      # A command will be executed by odo, e.g.: 'socat -d -d tcp-listen:20002,reuseaddr,fork tcp:localhost:5858'
      hostPort: 20002

# ... Omitted for brevity
```

In this scope, port-forwarding from 20002 to 5858 for example will be equivalent of executing
a socat command in this helper container, like so:

socat -d tcp-listen:20002,reuseaddr,fork tcp:localhost:5858

In the command above, this will open up port 20001 on the helper container,
and forwarding requests to localhost:5858
(which would be in the 'runtime' container, part of the same Pod)

[1] https://github.com/redhat-developer/odo/issues/6510

* Inject the right portForward client depending on the platform

* Delegate port-forwarding on Podman to the appropriate client

* Implement --forward-localhost on Podman

* Add unit and integration test cases

* Cover more debug-related test cases on Podman

* Expose the endpoint protocol so as to instruct socat to listen and forward the right protocol

* Fix sub-deps of EXEC and PORT_FORWARD, as suggested in review
2023-03-06 08:12:22 +01:00
Parthvi Vala
775adfd01e Support exec command (#6579)
* Support exec command for deploy

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Print log after timeout

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add helper function to form proper commandLine

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Mockgen kclient

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Enhance error message

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Attempt at fixing unit test failures

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Rename import v1 to batchv1

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Remove TODOs

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add integration tests and cleanup on user interrupt

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Temp changes

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Log tip to run odo logs after a minute

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* List components to delete even if there are no devfile resources

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix integration tests

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix deploy exec delete integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Temp Change

* Fix delete command tests

* Fix mockgen client

* Fix validation errors

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix unit test failure

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Attemp at writing less flaky integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Remove TODOs

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add tip after 1 minute and return the go routine if job finishes before that

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use the container as it is so that container-overrides can be taken into account

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Move job spec code to a different helper function inside the libdevfile pkg

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Modify the Execute method to use the new helper function and refactoring

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Attempt at fixing integration and unit tests

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Move defer to print remaining resources to a separate function, fix func doc and gofmt the files

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix test failures

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Cleanup

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Cleanup unused functions

Signed-off-by: Parthvi Vala <pvala@redhat.com>

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
2023-03-03 11:28:16 +01:00
Armel Soro
3adf8e5243 On Podman, detect if application is listening on the loopback interface, and either error out or not depending on --ignore-localhost (#6620)
* Add functions allowing to detect ports opened in a given container

Specifically, this will be useful in Podman to detect
applications that are bound to the loopback interface

* Make `odo dev` fail on Podman if we detect that the application is bound to the loopback interface (on any ports supposed to be forwarded)

Next step will be to provide an option for end-users
to override this behavior, by either:
- ignoring this error (--ignore-localhost);
- or explicitly adding a redirect via a side container (--forward-localhost)

More context in https://github.com/redhat-developer/odo/issues/6510#issuecomment-1439986558

* Add '--ignore-localhost' flag to 'odo dev' on Podman

Currently, `odo dev` on Podman will error out
if it detects that the application is listening on the container loopback interface.
Instead of erroring out, this flag allows users to ignore such failure; a warning will be displayed anyway if
the application is listening on the container loopback interface, but odo will not error out.
Ports will be marked as forwarded, but Podman might fail to redirect traffic to the application
if it is bound to this loopback interface.

* Add test cases

* Fix existing integration tests by passing --ignore-localhost on Podman

- odo describe component
- odo dev --debug

Some projects used there are listening to the loopback interface,
so they won't work on Podman unless --ignore-localhost is passed.

Next, we'll pass --forward-localhost when it is implemented,
so we can have a fully working project with port-forwarding.

* Extract logic for handling loopback ports in a separate method

Requested in review
2023-03-01 17:16:57 +01:00
Philippe Martin
99d6450499 Allow : character in devfile registry URI (#6624)
* Allow `:` character in devfile registry URI

* Add unit test
2023-02-27 18:32:11 +01:00
Philippe Martin
79504e3663 Describe non-odo component (#6615)
* Do not add odo/app on selector for named describe

* Display route

* Add integration tests

* Fix unit test
2023-02-27 13:41:45 +01:00
Armel Soro
7b9f214299 Bump Go to 1.19 (#6586)
* Set Go version in go.mod

go mod edit -go=1.19

* Fix formatting issues reported by gofmt

* Fix SA1019 check (usage of deprecated "io/ioutil"), reported by golangci-lint

SA1019: "io/ioutil" has been deprecated since Go 1.16:
As of Go 1.16, the same functionality is now provided by package io or package os,
and those implementations should be preferred in new code.
See the specific function documentation for details. (staticcheck)

* Use Go 1.19 in our Dockerfiles

* Use Go 1.19 in the rpm-prepare.sh script

* Update the tag for the IBM Cloud CI image
2023-02-16 09:03:48 -05:00
Armel Soro
262d11e3ef Bump version to v3.7.0 (#6596)
* Bump version to v3.7.0

* Update odo version in sample outputs diplayed in the documentation
2023-02-15 08:34:25 -05:00
Parthvi Vala
ad77fe77a3 Support for image component on podman (#6577)
* Add support for Image component on podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <asoro@redhat.com>

* Add integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <asoro@redhat.com>

* Add check for image list in the integration tests

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <asoro@redhat.com>

* Modify the build image msg

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <asoro@redhat.com>

* Add a note about the new envvar ODO_PUSH_IMAGES

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <asoro@redhat.com>
Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Update pkg/dev/podmandev/command.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update docs/website/docs/overview/configure.md

Co-authored-by: Armel Soro <armel@rm3l.org>

* fixup! Add integration test

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <asoro@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
2023-02-08 13:03:48 -05:00
Armel Soro
971496757b Support dev.odo.push.path:* attributes on Podman (#6576)
* Pick the right command attributes when searching for attributes prefixed with 'dev.odo.push.path'

We were looking into the run command attributes only,
while all this could also apply to the debug command too.

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* Test 'dev.odo.push.path' attribute using debug commands

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* Extract GetSyncFilesFromAttributes as an exported function in adapters

This way, it can be reused in all platform-specific implementations.

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* Support 'dev.odo.push.path' attribute on Podman

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* Update documentation

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* fixup! Extract GetSyncFilesFromAttributes as an exported function in adapters

---------

Co-authored-by: Parthvi Vala <pvala@redhat.com>
2023-02-08 05:46:52 -05:00
Armel Soro
162a193130 Fix UX inconsistency when handling commands bound to events (#6574)
* Do not display the full command-line when handling exec commands and no msg is specified

Displaying it adds a lot of unnecessary clutter.

* Display a shorter message when handling commands bound to 'pre-stop' events

This is to be more consistent with how the other messages are displayed.

* Display a shorter message when handling commands bound to 'post-start' events

This is to be more consistent with how the other messages are displayed.

* Fix integration tests

* fixup! Fix integration tests
2023-02-06 23:13:14 -05:00
Philippe Martin
704ff2a161 odo logs: Do not panic when no access to cluster/podman (#6561)
* odo logs: Do not panic when no access to cluster/podman

* Integration tests

* Remove unnecessary code

* Use defined errors
2023-02-06 12:11:20 +01:00
Armel Soro
eeda644cc9 Add support for OpenShift Devfile components (#6548)
* Add integration test case

Co-authored-by: Anand Kumar Singh <anandrkskd@gmail.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Add ApplyOpenShift method to handler

* Test openhift component with odo dev

* Rename GetKubernetesComponentsToPush to GetK8sAndOcComponentsToPush and modify if to obtain both k8s and oc components

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix unit test failures with delete_test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* update ListClusterResourcesToDeleteFromDevfile to fetch openshift component,Add ListOpenShiftComponents

Signed-off-by: anandrkskd <anandrkskd@gmail.com>

* fix testcase 'should have deleted the old resource and created the new resource' and add helper function ReplaceStrings

Signed-off-by: anandrkskd <anandrkskd@gmail.com>

* fix debug test to check openshift component

Signed-off-by: anandrkskd <anandrkskd@gmail.com>

* Update GetBindingsFromDevfile to include openshift components

* Update offline tests

* Add openshift component to devfiles

* Update tests

* Fix binding tests

* Fix RemoveBinding unit tests

* Handle OpenShift components when removing binding

* odo describe component displaysOpenShift components

* Remove unused function

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Signed-off-by: anandrkskd <anandrkskd@gmail.com>
Co-authored-by: Anand Kumar Singh <anandrkskd@gmail.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>
Co-authored-by: Parthvi Vala <pvala@redhat.com>
2023-02-03 14:30:24 +01:00
Parthvi Vala
d1fa996a25 Log error when podman client cannot be initialized (#6538)
* Log error when podman client cannot be initialized

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Return error if platform is podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Refactor errors related to cluster/podman inaccessible

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* fix validation failure

* rm3l's review

* Attempt at fixing CI failures

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Log error when unable to find podman client in NewPodmanCli

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Update pkg/podman/podman.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update pkg/podman/errors.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Fix integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
2023-02-03 09:47:58 +01:00
Philippe Martin
a94a5feaf7 Implement odo logs on podman (#6550)
* Implement odo logs for podman

* Test odo logs on podman

* Update pkg/podman/logs.go

Co-authored-by: Armel Soro <armel@rm3l.org>

---------

Co-authored-by: Armel Soro <armel@rm3l.org>
2023-01-31 12:38:33 -05:00
Philippe Martin
0bce606cfc Set platform and platformVersion in telemetry (#6540)
* Set platform and platformVersion in telemetry

* Vendor oc config client

* Get OC version

* Integration tests for odo dev

* Integration tests for odo deploy

* Update doc

* Apply suggestions from code review

Co-authored-by: Armel Soro <armel@rm3l.org>

* Add integration test on odo init

* Fix OC version when no complete version status

---------

Co-authored-by: Armel Soro <armel@rm3l.org>
2023-01-28 11:20:56 -05:00