1190 Commits

Author SHA1 Message Date
Armel Soro
cc2e8def81 Support autoBuild and deployByDefault on Image and Kubernetes/OpenShift components (#6654)
* Add sample Devfile with multiple autoBuild/deployByDefault combinations

It will be used for integration tests.

* Add helper function to update a given Devfile Command Group

This will allow supporting cases where we need to run a custom command,
but this is not implemented yet on odo (cases with 'odo dev --debug'
and 'odo deploy').
In this case, this helper will allow updating the Devfile for example to
unmark the current default command as non-default, and mark the custom
one as default.

* Add test cases for 'odo dev'

* Add test cases for 'odo deploy'

* Add test cases for 'odo build-images'

'odo build-images' should build all images regardless of the 'autoBuild' property.

* Display the spinner when creating or updating Kubernetes resources

This helps understand what resources are being patched.

* Handle deployByDefault on K8s and OpenShift components

* Add 'devfile.GetImageComponentsToPush' functions that returns the list of image components to auto-create

See https://github.com/devfile/api/issues/852#issuecomment-1211928487 for more context.

* Handle automatic image component creation for 'odo dev' on Kubernetes

* Handle automatic image component creation for 'odo dev' on Podman

* Handle automatic image and K8s/OpenShift component creation for 'odo deploy'

* Bump Devfile library to the latest commit at this time (c1b23d2)

This includes the fix for [1], which provides a way not to set default values automatically

[1] https://github.com/devfile/api/issues/1067

* Do not set default values when parsing a Devfile

See [1] for the rationale

[1] https://github.com/redhat-developer/odo/issues/5694\#issuecomment-1465778398

* Add documentation in the Devfile reference page

* Revert "Display the spinner when creating or updating Kubernetes resources"

This reverts commit 6ad073e63cb0e685f165eed767619a90997393a3.

* Avoid re-applying Image components multiple times

'adapter#Push' might get called several times when there are
state transitions (either on the Deployment in the cluster,
or from 'odo').
It might be confusing to apply Image components over and over again
(and also it can be slower if we need to push images to remote registries).

* Move GetK8sAndOcComponentsToPush and GetImageComponentsToPush to libdevfile package

As suggested in review, this should be the responsibility of the devfile library

Co-authored-by: Philippe Martin <phmartin@redhat.com>

* fixup! Handle automatic image and K8s/OpenShift component creation for 'odo deploy'

* fixup! Handle automatic image component creation for 'odo dev' on Podman

* fixup! Avoid re-applying Image components multiple times

* Apply suggestions from code review

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* fixup! Do not set default values when parsing a Devfile

* Fix devfile-deploy-functional-pods.yaml (used in 'odo logs' tests)

Set deployByDefault to false on innerloop-pod component.
Otherwise, since it is not referenced by any apply command,
it will be automatically created by *both* 'odo dev' and 'odo deploy'.

---------

Co-authored-by: Philippe Martin <phmartin@redhat.com>
Co-authored-by: Parthvi Vala <pvala@redhat.com>
2023-04-05 01:30:30 -04:00
Parthvi Vala
8876e1a579 Exit a test early if a session has exited (#6711)
Signed-off-by: Parthvi Vala <pvala@redhat.com>
2023-04-04 05:58:55 -04:00
Philippe Martin
252a77dde7 Speedup odo registry --details (#6678)
* Do not use deprecated DefaultStarterProjects

* Fix integration tests

* Use CommandGroups from registry index

* Fix JSON field name / Detailed err message

* Clarify --details is to be used with --devfile

* Document versions.commandGroups in json output
2023-04-03 10:05:04 -04:00
Philippe Martin
235539dfe9 Get DEVFILE_REGISTRY value for tests, to use local Registry (#6705)
* Get DEVFILE_REGISTRY value for tests

* Pass DEVFILE_REGISTRY to Windows tests

* Add values for devfile registry Helm deployment
2023-03-31 12:13:35 -04:00
Parthvi Vala
dfb36ddc56 odo dev/deploy should display a warning about default namespace on cluster (#6688)
* odo dev/deploy should display a warning about default namespace on cluster

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Move warning to after odo title

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add integration test for odo dev/deploy when not connected to a cluster or podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix podman test failure-s

* Add a new line before warning, and use corev1.NamespaceDefault instead of default

Signed-off-by: Parthvi Vala <pvala@redhat.com>

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
2023-03-29 09:36:15 +02:00
Parthvi Vala
61d53dc99c Add name detection to odo analyze (#6682)
* Add name detection to odo analyze

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Update tests/helper/helper_generic.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update tests/integration/cmd_analyze_test.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update odo analyze documentation

Signed-off-by: Parthvi Vala <pvala@redhat.com>

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
2023-03-28 16:57:29 +02:00
Philippe Martin
163c164d18 Pod Security Admission support for odo deploy (#6679)
* Use GetPodTemplateSpec instead of deprecated GetContainers

* Specify pod admission policy when building Exec Job

* Integration test

* Update tests/helper/component_podman.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* add test

---------

Co-authored-by: Armel Soro <armel@rm3l.org>
2023-03-28 16:23:11 +02:00
Parthvi Vala
c70aaff642 Fix: odo init does not ask for starter project if the Devfile stack contains extra files (#6684)
Signed-off-by: Parthvi Vala <pvala@redhat.com>
2023-03-27 11:18:17 -04:00
Parthvi Vala
47234b94b3 Fix: Starterproject not replace existing devfile stack files of the context dir (#6633)
* Starterproject does not replace existing files of the context dir

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix issue with os.RemoveAll

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix errors, it works now

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix test failures

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use filesystem to perform file operations wherever possible

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use filesystem to perform file operations wherever possible (2)

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add unit test for DownloadStarterProject 1/n

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix zip files

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix zip files (2)

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add unit test starter project with Devfile

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix zip files (3)

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Finish unit tests

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Remove unnecessary integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add comments and verbosity, cleanup code

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Update pkg/registry/registry.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update pkg/util/util.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update pkg/registry/registry.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* rm3l's suggestion on using err when not shadowed and enhance error messages

Signed-off-by: Parthvi Vala <pvala@redhat.com>

Fix validation errors

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Change arg position in IsValidProjectDir

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* rename isConflicting to getConflictingFiles

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add zip files for starterproject with empty dir

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add unit tests covering starterprojects with empty dir for both conflicting and non-conflicting cases

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Handle error while walking

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Delete temp dirs

Signed-off-by: Parthvi Vala <pvala@redhat.com>

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
2023-03-21 08:55:42 -04:00
Philippe Martin
93d1330182 Patch Pods to pass pod security admission (#6602)
* Bump devfile/library

* Use generator.GetPodTemplateSpec instead of Get[Init]Containers

* Fix deprecated pointer.*

* Fix more deprecated

* Workaround for ServerPreferredResources panicking

* Pass policy to GetPodTemplateSpec

* Integration test

* document behaviour

* Review
2023-03-20 10:16:14 -04:00
Armel Soro
69a83832d7 Fix potential issue with 'odo describe component' integration tests when Podman is installed (#6642)
When running locally with Podman installed,
the error message no longer contains the reference to the namespace.
2023-03-08 14:06:33 +01:00
Armel Soro
b3bc7c47f5 Skip DevfileRegistriesList-related tests on non Kubernetes clusters (#6640)
On Prow, users might not have the permission to install the CRDs.
It is actually fine to execute these tests only on Kubernetes.
2023-03-07 16:21:49 +01:00
Philippe Martin
26c90d7c4d Remove podman from experimental mode (#6628)
* Remove podman from experimental mode

* Remove env var for odo dev

* Cleanup tests and sources about experimental mode

* Hide "Experimental mode" doc page

* Cleanup latest commit

* doc odo dev --platform podman

* Doc other commands

* Apply suggestions from code review

Co-authored-by: Armel Soro <armel@rm3l.org>

* Remove more ODO_EXPERIMENTAL_MODE from latest commits on main

* Remove reference to Experimental mode from blog posts

---------

Co-authored-by: Armel Soro <armel@rm3l.org>
2023-03-06 14:17:39 +01:00
Armel Soro
4fb7308cfa Automatically use in-cluster Devfile registries if any (#6622)
* Add kubeclient as dependency of the registry client

Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Add GetRegistryList method to the kube client interface

This is yet to be implemented.

Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Implement GetRegistryList

* adding test if devfileRegistryListCR is present in cluster

Signed-off-by: anandrkskd <anandrkskd@gmail.com>

* Unit tests (to be continued)

* Add unit test cases against kclient#GetRegistryList()

Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Ignore in-cluster registries with an empty URL

This should ideally not happen if the registry operator
 is installed in the cluster (because it validates the
 URL to make sure it is reachable), but you never know ;-)

* Update error message when trying to remove registry

Registries might be found in the cluster.

* Pass isSecure value to the registry handler

* Make it possible to use in-cluster registries when calling 'odo registry --details'

* Remove unused 'preferenceClient' from registry.getRegistryStacks

* Handle in-cluster registries in 'odo init' non-interactive mode

* Handle in-cluster registries in 'odo init' interactive mode

* Add integration test for odo init --devfile-registry

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use proxy when available

Signed-off-by: Parthvi Vala <pvala@redhat.com>

Co-authored-by: Armel Soro <armel@rm3l.org>

* Make sure tests work even if the registry operator is installed in the cluster or if there are cluster-wide registry lists

* Add tests for 'odo init' interactive mode

* Remove useless CR file

CRs are now dynamically created and applied from the tests

* fixup! Add tests for 'odo init' interactive mode

---------

Signed-off-by: anandrkskd <anandrkskd@gmail.com>
Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>
Co-authored-by: anandrkskd <anandrkskd@gmail.com>
Co-authored-by: Parthvi Vala <pvala@redhat.com>
2023-03-06 09:19:02 +01:00
Armel Soro
29b5a38f88 Make port-forwarding work on Podman with apps listening on the loopback interface, via a new --forward-localhost flag (#6629)
* Embed platform.Client interface in platform-specific interfaces

This avoids repeating the same methods in both interfaces,
and makes the intent clearer.

* Verify interface compliance of PodmanCli at compile time

This is recommended in the Coding Conventions guidelines [1].
Specifically, what's important here is checking that it meets the 'platform.Client' contract.

[1] https://github.com/redhat-developer/odo/wiki/Dev:-Coding-Conventions#verify-interface-compliance

* Move K8s-specific implementation of port-forwarding to a dedicated package

This paves the way to providing a different implementation for Podman

* Remove GetPortsToForward method from the portForward.Client interface

Current implementation relies on the Devfile object,
so it makes more sense to be in the libdevfile package.

* Monitor and send appropriate status events after starting a remote command process

This allows callers to get more meaningful events about the process.

* Implement port-forwarding logic on Podman

As explained in [1], this makes use of a helper sidecar container
(aptly named "odo-helper-port-forwarding") to be added to the Pod Spec created by odo.
In this scope, port-forwarding will be equivalent of executing a socat command in this
helper container, like so:

socat -d tcp-listen:20002,reuseaddr,fork tcp:localhost:5858

In the command above, this will open up port 20001 on the helper container,
and forwarding requests to localhost:5858
(which would be in the application container, part of the same Pod)

[1] https://github.com/redhat-developer/odo/issues/6510

* Update portForward.Client interface methods

Specifically, the 'StartPortForwarding' method
can now accept an explicit list of ports that needs to
be forwarded, if the caller can compute provide such information.

This is currently useful on Podman where the ports
(even the random ones) are known in advance.

* Add helper sidecar container to the Pod Spec generated on Podman

As explained in [1], this helper sidecar container (aptly named "odo-helper-port-forwarding")
will hold the actual container/host ports mapping in the spec
(to overcome the limitation of using hostPort against a container app listening on the loopback interface);
this running container will be used to execute the actual port-forwarding commands (based on socat), e.g:

```
apiVersion: v1
kind: Pod
metadata:
  name: my-component-app
  labels:
    app: my-component-app
spec:
  containers:
  - name: runtime
    command: [ 'tail', '-f', '/dev/null']
    # Omitted for brevity, but imagine an application being executed by odo
    # and listening on both 0.0.0.0:3000 and 127.0.0.1:5858

  - name: odo-helper-port-forwarding
    image: quay.io/devfile/base-developer-image:ubi8-latest
    command: [ 'tail', '-f', '/dev/null']
    ports:
    - containerPort: 20001
      # A command will be executed by odo, e.g.: 'socat -d -d tcp-listen:20001,reuseaddr,fork tcp:localhost:3000'
      hostPort: 20001
    - containerPort: 20002
      # A command will be executed by odo, e.g.: 'socat -d -d tcp-listen:20002,reuseaddr,fork tcp:localhost:5858'
      hostPort: 20002

# ... Omitted for brevity
```

In this scope, port-forwarding from 20002 to 5858 for example will be equivalent of executing
a socat command in this helper container, like so:

socat -d tcp-listen:20002,reuseaddr,fork tcp:localhost:5858

In the command above, this will open up port 20001 on the helper container,
and forwarding requests to localhost:5858
(which would be in the 'runtime' container, part of the same Pod)

[1] https://github.com/redhat-developer/odo/issues/6510

* Inject the right portForward client depending on the platform

* Delegate port-forwarding on Podman to the appropriate client

* Implement --forward-localhost on Podman

* Add unit and integration test cases

* Cover more debug-related test cases on Podman

* Expose the endpoint protocol so as to instruct socat to listen and forward the right protocol

* Fix sub-deps of EXEC and PORT_FORWARD, as suggested in review
2023-03-06 08:12:22 +01:00
Parthvi Vala
775adfd01e Support exec command (#6579)
* Support exec command for deploy

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Print log after timeout

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add helper function to form proper commandLine

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Mockgen kclient

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Enhance error message

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Attempt at fixing unit test failures

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Rename import v1 to batchv1

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Remove TODOs

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add integration tests and cleanup on user interrupt

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Temp changes

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Log tip to run odo logs after a minute

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* List components to delete even if there are no devfile resources

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix integration tests

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix deploy exec delete integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Temp Change

* Fix delete command tests

* Fix mockgen client

* Fix validation errors

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix unit test failure

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Attemp at writing less flaky integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Remove TODOs

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add tip after 1 minute and return the go routine if job finishes before that

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use the container as it is so that container-overrides can be taken into account

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Move job spec code to a different helper function inside the libdevfile pkg

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Modify the Execute method to use the new helper function and refactoring

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Attempt at fixing integration and unit tests

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Move defer to print remaining resources to a separate function, fix func doc and gofmt the files

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix test failures

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Cleanup

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Cleanup unused functions

Signed-off-by: Parthvi Vala <pvala@redhat.com>

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
2023-03-03 11:28:16 +01:00
ritudes
ff920107f9 Fix failing tests due to notifications related to update in output (#6625)
* Fix failing tests due to notifications related to update in output

Signed-off-by: Ritu Deshmukh <rideshmu@redhat.com>

* less strict check

Signed-off-by: anandrkskd <anandrkskd@gmail.com>

* Update tests/integration/cmd_devfile_build_images_test.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update tests/integration/cmd_devfile_build_images_test.go

Co-authored-by: Armel Soro <armel@rm3l.org>

---------

Signed-off-by: Ritu Deshmukh <rideshmu@redhat.com>
Signed-off-by: anandrkskd <anandrkskd@gmail.com>
Co-authored-by: anandrkskd <anandrkskd@gmail.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
2023-03-03 09:47:49 +01:00
Armel Soro
3adf8e5243 On Podman, detect if application is listening on the loopback interface, and either error out or not depending on --ignore-localhost (#6620)
* Add functions allowing to detect ports opened in a given container

Specifically, this will be useful in Podman to detect
applications that are bound to the loopback interface

* Make `odo dev` fail on Podman if we detect that the application is bound to the loopback interface (on any ports supposed to be forwarded)

Next step will be to provide an option for end-users
to override this behavior, by either:
- ignoring this error (--ignore-localhost);
- or explicitly adding a redirect via a side container (--forward-localhost)

More context in https://github.com/redhat-developer/odo/issues/6510#issuecomment-1439986558

* Add '--ignore-localhost' flag to 'odo dev' on Podman

Currently, `odo dev` on Podman will error out
if it detects that the application is listening on the container loopback interface.
Instead of erroring out, this flag allows users to ignore such failure; a warning will be displayed anyway if
the application is listening on the container loopback interface, but odo will not error out.
Ports will be marked as forwarded, but Podman might fail to redirect traffic to the application
if it is bound to this loopback interface.

* Add test cases

* Fix existing integration tests by passing --ignore-localhost on Podman

- odo describe component
- odo dev --debug

Some projects used there are listening to the loopback interface,
so they won't work on Podman unless --ignore-localhost is passed.

Next, we'll pass --forward-localhost when it is implemented,
so we can have a fully working project with port-forwarding.

* Extract logic for handling loopback ports in a separate method

Requested in review
2023-03-01 17:16:57 +01:00
HarshithaMS005
463c15d212 Use Node image compatible with Z and P environments (#6626)
* nodejs-compatible-image-to-resolve-podman-testcases-issues

* Update Dockerfile

* Update Dockerfile

* Update Dockerfile

---------

Co-authored-by: HarshithaMS <M.S.Harshitha@ibm.com>
2023-03-01 13:51:11 +01:00
Philippe Martin
79504e3663 Describe non-odo component (#6615)
* Do not add odo/app on selector for named describe

* Display route

* Add integration tests

* Fix unit test
2023-02-27 13:41:45 +01:00
ritudes
b817365d5b Adding check in odo dev interactive mode for pod/sync to complete (#6614) 2023-02-27 09:30:48 +01:00
ritudes
188d7e1548 Adding ritudes user id to tests/OWNERS (#6603)
* Adding my id to OWNERS file

* Adding ritudes-user id to scripts/OWNERS
2023-02-17 11:59:53 +01:00
Armel Soro
7b9f214299 Bump Go to 1.19 (#6586)
* Set Go version in go.mod

go mod edit -go=1.19

* Fix formatting issues reported by gofmt

* Fix SA1019 check (usage of deprecated "io/ioutil"), reported by golangci-lint

SA1019: "io/ioutil" has been deprecated since Go 1.16:
As of Go 1.16, the same functionality is now provided by package io or package os,
and those implementations should be preferred in new code.
See the specific function documentation for details. (staticcheck)

* Use Go 1.19 in our Dockerfiles

* Use Go 1.19 in the rpm-prepare.sh script

* Update the tag for the IBM Cloud CI image
2023-02-16 09:03:48 -05:00
Parthvi Vala
ad77fe77a3 Support for image component on podman (#6577)
* Add support for Image component on podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <asoro@redhat.com>

* Add integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <asoro@redhat.com>

* Add check for image list in the integration tests

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <asoro@redhat.com>

* Modify the build image msg

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <asoro@redhat.com>

* Add a note about the new envvar ODO_PUSH_IMAGES

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <asoro@redhat.com>
Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Update pkg/dev/podmandev/command.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update docs/website/docs/overview/configure.md

Co-authored-by: Armel Soro <armel@rm3l.org>

* fixup! Add integration test

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <asoro@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
2023-02-08 13:03:48 -05:00
Armel Soro
971496757b Support dev.odo.push.path:* attributes on Podman (#6576)
* Pick the right command attributes when searching for attributes prefixed with 'dev.odo.push.path'

We were looking into the run command attributes only,
while all this could also apply to the debug command too.

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* Test 'dev.odo.push.path' attribute using debug commands

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* Extract GetSyncFilesFromAttributes as an exported function in adapters

This way, it can be reused in all platform-specific implementations.

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* Support 'dev.odo.push.path' attribute on Podman

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* Update documentation

Co-authored-by: Parthvi Vala <pvala@redhat.com>

* fixup! Extract GetSyncFilesFromAttributes as an exported function in adapters

---------

Co-authored-by: Parthvi Vala <pvala@redhat.com>
2023-02-08 05:46:52 -05:00
Armel Soro
162a193130 Fix UX inconsistency when handling commands bound to events (#6574)
* Do not display the full command-line when handling exec commands and no msg is specified

Displaying it adds a lot of unnecessary clutter.

* Display a shorter message when handling commands bound to 'pre-stop' events

This is to be more consistent with how the other messages are displayed.

* Display a shorter message when handling commands bound to 'post-start' events

This is to be more consistent with how the other messages are displayed.

* Fix integration tests

* fixup! Fix integration tests
2023-02-06 23:13:14 -05:00
Philippe Martin
704ff2a161 odo logs: Do not panic when no access to cluster/podman (#6561)
* odo logs: Do not panic when no access to cluster/podman

* Integration tests

* Remove unnecessary code

* Use defined errors
2023-02-06 12:11:20 +01:00
Armel Soro
eeda644cc9 Add support for OpenShift Devfile components (#6548)
* Add integration test case

Co-authored-by: Anand Kumar Singh <anandrkskd@gmail.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Add ApplyOpenShift method to handler

* Test openhift component with odo dev

* Rename GetKubernetesComponentsToPush to GetK8sAndOcComponentsToPush and modify if to obtain both k8s and oc components

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix unit test failures with delete_test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* update ListClusterResourcesToDeleteFromDevfile to fetch openshift component,Add ListOpenShiftComponents

Signed-off-by: anandrkskd <anandrkskd@gmail.com>

* fix testcase 'should have deleted the old resource and created the new resource' and add helper function ReplaceStrings

Signed-off-by: anandrkskd <anandrkskd@gmail.com>

* fix debug test to check openshift component

Signed-off-by: anandrkskd <anandrkskd@gmail.com>

* Update GetBindingsFromDevfile to include openshift components

* Update offline tests

* Add openshift component to devfiles

* Update tests

* Fix binding tests

* Fix RemoveBinding unit tests

* Handle OpenShift components when removing binding

* odo describe component displaysOpenShift components

* Remove unused function

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Signed-off-by: anandrkskd <anandrkskd@gmail.com>
Co-authored-by: Anand Kumar Singh <anandrkskd@gmail.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>
Co-authored-by: Parthvi Vala <pvala@redhat.com>
2023-02-03 14:30:24 +01:00
Parthvi Vala
d1fa996a25 Log error when podman client cannot be initialized (#6538)
* Log error when podman client cannot be initialized

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Return error if platform is podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Refactor errors related to cluster/podman inaccessible

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* fix validation failure

* rm3l's review

* Attempt at fixing CI failures

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Log error when unable to find podman client in NewPodmanCli

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Update pkg/podman/podman.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Update pkg/podman/errors.go

Co-authored-by: Armel Soro <armel@rm3l.org>

* Fix integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

---------

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
2023-02-03 09:47:58 +01:00
Philippe Martin
e0ab28c181 Test version upon oc/kubectl version (#6551)
* Test version upon oc/kubectl version

* Fix test when version is not accessible due to permissions
2023-01-31 14:22:09 -05:00
Philippe Martin
a94a5feaf7 Implement odo logs on podman (#6550)
* Implement odo logs for podman

* Test odo logs on podman

* Update pkg/podman/logs.go

Co-authored-by: Armel Soro <armel@rm3l.org>

---------

Co-authored-by: Armel Soro <armel@rm3l.org>
2023-01-31 12:38:33 -05:00
Philippe Martin
0bce606cfc Set platform and platformVersion in telemetry (#6540)
* Set platform and platformVersion in telemetry

* Vendor oc config client

* Get OC version

* Integration tests for odo dev

* Integration tests for odo deploy

* Update doc

* Apply suggestions from code review

Co-authored-by: Armel Soro <armel@rm3l.org>

* Add integration test on odo init

* Fix OC version when no complete version status

---------

Co-authored-by: Armel Soro <armel@rm3l.org>
2023-01-28 11:20:56 -05:00
Armel Soro
34ecccdf11 Fix potential issues with E2E tests (#6543)
* Make 'helper.Component#Exec' accept an additional parameter that allows checking (or not) the remote command execution

* In E2E tests, wait until the app is completely ready before sending requests to the local forwarded port

Otherwise, sending a request too early causes
"connection refused" errors, which causes port-forwarding to be restarted
(with new local ports because "odo dev" is running with "--random-ports").
As a first start, here we are making sure that the application port is available in the container,
prior to sending requests to the local forwarded port. This should reduce the likeliness
of port-forwarding restarts.

In the future, we might want to make the test detect the new local ports and use those instead.
We could even make sure to make "odo dev" reuse the previous random ports it had, provided
they are available again.

* Increase timeout and polling interval when waiting for port-forward restart

* Add a '/ping' endpoint to the sample Go app used in E2E

This allows checking the application running in the container.

* Do not stop the DevSession process before removing the Binding from the Devfile

https://github.com/redhat-developer/odo/issues/6101 has been fixed.

* fixup! Make 'helper.Component#Exec' accept an additional parameter that allows checking (or not) the remote command execution

* fixup! In E2E tests, wait until the app is completely ready before sending requests to the local forwarded port

* Create reusable 'helper.WaitAppReadyInContainer' helper function, as suggested in review

Co-authored-by: Philippe Martin <phmartin@redhat.com>

---------

Co-authored-by: Philippe Martin <phmartin@redhat.com>
2023-01-27 13:26:27 -05:00
Armel Soro
9a39462615 Show information about forwarded debug ports in odo describe component (#6537)
* Refactor 'libdevfile#GetContainerEndpointMapping' so it returns the whole Endpoint struct

It will help extract useful information like the port name and exposure.

* Fix typo in parameter name (in portForward#Client interface)

* Display the port names in the output of 'odo describe component'

* Display the port exposures in the output of 'odo describe component'

* Tell whether the port is a debug one or not in the output of 'odo describe component'

* Add integration tests meeting the expectations

* Update documentation
2023-01-27 12:02:57 -05:00
Armel Soro
87ef5645d4 Make sure to delete the component resources after each "delete component" test spec (#6541)
* Make sure to fully delete the component resources after each "delete component" test spec

Otherwise, "odo delete component --running-in $mode" might leave some resources running
(on Podman especially).
For example, the Podman test that runs "delete component --running-in deploy"
(after starting and killing a Dev Session abruptly) might leave those Dev resources running.
This has been alleviated in CI via 9ebf766 by ensuring Ginkgo does not hang indefinitely and
by deleting those leftover resources.
But the issue was still persisting locally.

* Do not error out if there is no pod to stop in the last Podman test job step
2023-01-26 04:33:06 -05:00
Philippe Martin
facfb79cca Run Docs tests on CI/CD (#6521)
* Run Docs tests on CI/CD

* Remove warnings from test output

* Replace version in mdx files
2023-01-25 13:50:02 -05:00
Parthvi Vala
e3e8b341ef Display error message when odo dev fails on podman and clean resources (#6522)
* Display error message when odo dev fails on podman and clean resources on error

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* show complete podman output on error

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Only set deployedPod if it exists

Signed-off-by: Parthvi Vala <pvala@redhat.com>

Signed-off-by: Parthvi Vala <pvala@redhat.com>
2023-01-24 07:40:48 -05:00
Armel Soro
83d98c2a52 Implement odo delete component --running-in (#6485)
* Add '--running-in' CLI flag to 'odo delete component'

* Use the provided 'running-in' to filter resources candidate for deletion on both cluster and Podman

* Update 'odo delete component' confirmation and success messages to mention the provided value for 'running-in'

* Add unit test cases

* Add integration test cases

* Extract sample outputs from delete-component command reference docs

* Add documentation for the '--running-in' flag
2023-01-23 17:50:22 -05:00
Parthvi Vala
19f8e0ebdb Update devfile/library to support pod-overrides and container-overrides attributes and add integration test for it (#6512)
* Update the Devfile library

* Add test for container-overrides and pod-overrides attributes

Signed-off-by: Parthvi Vala <pvala@redhat.com>

Attempt at fixing CI failures

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix CI failure

* Attempt at fixing OC integration test failures

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use random name in integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Rebase and fix integration test failure

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Make integration test work for podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Temp attempt at fixing podman test for GH

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Another attempt at fixing CI test for podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* One more attempt at fixing integration test for podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>

Signed-off-by: Parthvi Vala <pvala@redhat.com>
2023-01-23 15:21:18 -05:00
Philippe Martin
d2443d0fc0 Change port range used for port-forwarding endpoints to start at 20001 (#6519)
* Change port forward to start at 20001

* Update docs
2023-01-23 13:21:00 -05:00
Armel Soro
511792a62c Isolate Podman tests in namespaces (#6499)
* Rename SetProjectName into GetProjectName

Co-authored-by: Anand Singh <ansingh@redhat.com>
Co-authored-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Generate specific containers.conf file for each test spec using a dedicated engine namespace

Co-authored-by: Anand Singh <ansingh@redhat.com>
Co-authored-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Listen on random ports on Podman when '--random-ports' is used

This reduces the risks of port conflicts when running test specs in parallel.

Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Exclude Gosec G404 (use of math/rand) rule

Co-authored-by: Philippe Martin <phmartin@redhat.com>

* Run Podman specs in parallel

* Output the Pod spec to be played by Podman depending on verbosity level

This will help debug potential issues.

* Use random name in 'using devfile that contains K8s resource to run it on podman' test

* Use random component name in sample java-quarkus project used in 'a hotReload capable project is used with odo dev' test

* Revert "Run Podman specs in parallel"

Parallelization works great on GitHub Actions, but I experimented a lot
of issues when running locally with a lot (~11) of parallel test nodes.
Not sure why exactly, but some containers created by Podman had a lot of
networking issues.
We can look into parallelizing the runs later in a subsequent PR.

This reverts commit 64d5d31248a62f355a32ca245ba399a723fdb22f.

* Allow overridding the number of parallel nodes for Podman integration tests

This way, we could be able to run them in parallel on GitHub
but sequentially (default) locally.
This will still benefit us by reducing the time it takes to run such tests on GitHub.

Meanwhile, we can look into the issues we have locally with parallelization.

Note that it is still possible to run them locally in parallel via
the PODMAN_EXEC_NODE env var.

Co-authored-by: Anand Singh <ansingh@redhat.com>
Co-authored-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>
2023-01-23 09:25:57 -05:00
Philippe Martin
f089cba02a Set experimental mode in telemetry (#6520)
* Set experimental mode in telemetry

* Add integration tests
2023-01-23 06:47:43 -05:00
Philippe Martin
ae82d18928 Make doc tests pass (#6516)
* Fix docs

* Display env vars in a sorted order

* Run on quickstart-demo
2023-01-20 09:53:51 -05:00
Parthvi Vala
992c48a7d1 odo describe component: display remote source code location for each container component (#6497)
* Describe remote source code location for container component

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix cI failures

* Add extra check for .mountSources and add more integration tests

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use GetMountSources() instead of property

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Use random name in integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

Signed-off-by: Parthvi Vala <pvala@redhat.com>
2023-01-19 17:00:39 -05:00
Philippe Martin
b86afbc8eb Add more tests for odo dev on Podman (3) (#6515)
* Change DevModeShouldFail signature

* Use component.Exec for remoteFileChecker

* Test run failures on podman

* Test odo dev with custom run commands

* Test "Create and dev java-springboot component" on Podman as well

NOTE: Previous container image (quay.io/eclipse/che-java11-maven:next) was not working on Podman due to permission errors.
We changed the image to the same one used on the Devfile public registry: registry.access.redhat.com/ubi8/openjdk-11:latest

Example of output:
```
[odo] arrow_right_hook Running on podman in Dev mode
[odo]  •  Deploying pod  ...
 ✓  Deploying pod [35s]
[odo]  ✗  Command 'tar xf - -C /projects --no-same-owner' in container failed.
...
[odo]  ✗  failed to sync to component with name java-spring-boot: unable push files to pod: exit status 2
```

* Test "running odo dev --no-watch and build command throws an error" on Podman as well

* Test "running odo dev and run command throws an error" on Podman as well

* Test "running odo dev and prestart events are defined" on Podman as well

Co-authored-by: Armel Soro <asoro@redhat.com>
2023-01-19 12:43:55 -05:00
Philippe Martin
6d9bb37acb hotReloadCapable recognized on podman (#6509)
* Fix

* Add integration tests
2023-01-19 11:24:42 -05:00
Armel Soro
a7f6d96504 Add more tests for odo dev on Podman (2) (#6506)
* prepare test dev.odo.push.path - not implemented yet on Podman

* Skip 'should create vcs-uri annotation for the deployment when running odo dev' on Podman

* Test cpu/memory limit/resources on podman/cluster

* WIP add GetPodDef implementation for cluster

Signed-off-by: anandrkskd <anandrkskd@gmail.com>

* Convert springboot compare local and remote files test for podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Fix namespace instead of component name

* Change image and make adjustments for this image

* Remove cpu request/limit

* Fix rebase

Signed-off-by: anandrkskd <anandrkskd@gmail.com>
Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Philippe Martin <phmartin@redhat.com>
Co-authored-by: anandrkskd <anandrkskd@gmail.com>
Co-authored-by: Parthvi Vala <pvala@redhat.com>
2023-01-19 09:07:20 -05:00
Armel Soro
d468842a32 Port-forward Debug endpoints only when running odo dev with --debug (#6505)
* Add integration tests meeting the expectations

* Refactor 'libdevfile.GetContainerEndpointMapping' such that it returns debug endpoints only if told so

* [Kubernetes] Port-forward Debug endpoints only if running in Debug mode

* [Podman] Port-forward Debug endpoints only if running in Debug mode

* Use '--inspect' instead of '--inspect-brk' to run the "debug" script in Node.JS sample projects

'--inspect-brk' stops the execution at the start of the command
(waiting for a debugger to attach to it) while '--inspect' does not.
We need the application to be started regardless of whether a debugger
is attached or not, as we are testing that we can communicate with
the forwarded ports.

* Temporarily skip the test step on Podman that connects to the forwarded debug port until [1] is fixed

We can work on fixing the issue on Podman in a separate PR.

[1] https://github.com/redhat-developer/odo/issues/6510
2023-01-19 06:21:55 -05:00
Armel Soro
51ebf26dcb Make sure to use random component names when copying sample Devfiles in integration tests (#6504)
* Make sure to use unique random component names in integration tests

This will help prevent conflicts when isolation is not possible (for example on Podman)

For consistency, next step would be not to use 'odo init --devfile-path', but instead rely on the updated helper.CopyExampleDevfile function everywhere.

* fixup! Make sure to use unique random component names in integration tests

* Eventually check server response in 'should expose all endpoints on localhost regardless of exposure' test
2023-01-18 12:39:47 -05:00
Parthvi Vala
53a2a23b3a Ignore and show warning when handling Kubernetes components on Podman (#6471)
* Ignore and show warning when handling Kubernetes components on Podman

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Only use comp names

* Add integration test

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* return early when there are no k8s resources to be created; get resources referenced by apply command and add test for it

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Warn about k8s resources that match the given CommandGroupKind

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Only warn about k8s resources for a given command and groupkind

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Amend the test case

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add warning about image components referenced by an apply command and attempt at fixing CI failures

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Add warning inside ApplyImage and ApplyKubernetes methods

Signed-off-by: Parthvi Vala <pvala@redhat.com>

* Update pkg/dev/podmandev/reconcile.go

Co-authored-by: Armel Soro <armel@rm3l.org>

Signed-off-by: Parthvi Vala <pvala@redhat.com>
Co-authored-by: Armel Soro <armel@rm3l.org>
2023-01-17 23:28:35 -05:00