diff --git a/README.md b/README.md index e68a76e..a9d961f 100644 --- a/README.md +++ b/README.md @@ -31,6 +31,9 @@ - [Multi-node deployment](#multi-node-deployment) * [General information](#general-information) * [Setup](#setup) +- [Multi-host deployment](#multi-host-deployment) + * [General information](#general-information-1) + * [Setup](#setup-1) - [Configuration](#configuration) - [Theming](#theming) - [Troubleshoot](#troubleshoot) @@ -80,7 +83,7 @@ Isaiah has all these features implemented : - Support for custom Docker Host / Context. - Support for extensive configuration with `.env` - Support for HTTP and HTTPS -- Support for standalone / proxy / multi-node deployment +- Support for standalone / proxy / multi-node / multi-host deployment On top of these, one may appreciate the following characteristics : - Written in Go (for the server) and Vanilla JS (for the client) @@ -144,6 +147,8 @@ Here's a description of every example : - `docker-compose.agent.yml`: A sample setup with Isaiah operating as an Agent in a multi-node deployment. +- `docker-compose.host.yml`: A sample setup with Isaiah expecting to communicate with other hosts in a multi-host deployment. + When your `docker-compose` file is on point, you can use the following commands : ```sh # Option 1 : Run Isaiah in the current terminal (useful for debugging) @@ -299,6 +304,46 @@ If encounter any issue, please read the [Troubleshoot](#troubleshoot) section. > You may want to note that you don't need to expose ports on the machine / Docker container running Isaiah when it is configured as an Agent. +## Multi-host deployment + +Using Isaiah, you can manage multiple hosts with their own distinct Docker resources from a single dashboard. + +Before delving into that part, please get familiar with the general information below. + +### General information + +The big difference between multi-node and multi-host deployments is that you won't need to install Isaiah on every single node +if you are using multi-host. In this setup, Isaiah is installed only on one server, and communicates with other Docker hosts +directly over TCP / Unix sockets. It makes it easier to manage multiple remote Docker environments without having to setup Isaiah +on all of them. + +Please note that, in a multi-host setup, there must be a direct access between the main host (where Isaiah is running) +and the other ones. Usually, they should be on the same network, or visible through a secured gateway / VPN / filesystem mount. + +Let's see how to set up a multi-host deployment. + +### Setup + +In order to help you get started, a [sample file](/app/sample.docker_hosts) was created. + +First, please ensure the following : +- Your `Master` host is running, exposed on the network, and available in your web browser +- Your `Master` host has the setting `MULTI_HOST_ENABLED` set to `true`. +- Your `Master` host has access to the other Docker hosts over TCP / Unix socket. + +Second, please create a `docker_hosts` file next to Isaiah's executable, using the sample file cited above: +- Every line should contain two strings separated by a single space. +- The first string is the name of your host, and the second string is the path to reach it. +- The path to your host should look like this : [PROTOCOL]://[URI] +- Example 1 : Local unix:///var/run/docker.sock +- Example 2 : Remote tcp://my-domain.tld:4382 + +> If you're using Docker, you can mount the file at the root of the filesystem, as in :
+`docker ... -v my_docker_hosts:/docker_hosts ...` + +Finally, launch Isaiah on the Master host, and you should see logs indicating whether connection with remote hosts was established. +Eventually, you will see `Master` with `The name of your host` in the lower right corner of your screen. + ## Configuration To run Isaiah, you will need to set the following environment variables in a `.env` file located next to your executable : @@ -329,6 +374,7 @@ To run Isaiah, you will need to set the following environment variables in a `.e | `MASTER_HOST` | `string` | For multi-node deployments only. The host used to reach the Master node, specifying the IP address or the hostname, and the port if applicable (e.g. my-server.tld:3000). | Empty | | `MASTER_SECRET` | `string` | For multi-node deployments only. The secret password used to authenticate on the Master node. Note that it should equal the `AUTHENTICATION_SECRET` setting on the Master node. | Empty | | `AGENT_NAME` | `string` | For multi-node deployments only. The name associated with the Agent node as it is displayed on the web interface. It should be unique for each Agent. | Empty | +| `MULTI_HOST_ENABLED` | `boolean` | Whether Isaiah should be run in multi-host mode. When enabled, make sure to have your `docker_hosts` file next to the executable. | False | > **Note :** Boolean values are case-insensitive, and can be represented via "ON" / "OFF" / "TRUE" / "FALSE" / 0 / 1. @@ -421,7 +467,7 @@ I leave here a few ideas that I believe could be implemented, but may require mo - Wrap every command in a "block" (begin - command - end) to easily distinguish user-sent commands from output - Sending to the real terminal the key presses captured from the web (a.k.a sending key presses to a running process) -Ultimately, please also note that in a multi-node setup, the extra network latency and unexpected buffering from remote terminals may cause additional display artifacts. +Ultimately, please also note that in a multi-node / multi-host setup, the extra network latency and unexpected buffering from remote terminals may cause additional display artifacts. #### An error happens when spawning a new shell on the server / inside a Docker container diff --git a/app/client/assets/js/isaiah.js b/app/client/assets/js/isaiah.js index 64e8d5b..64419a3 100644 --- a/app/client/assets/js/isaiah.js +++ b/app/client/assets/js/isaiah.js @@ -537,6 +537,7 @@ bulk: 'Bulk actions', theme: 'Theme', agent: 'Agent', + host: 'Host', }[menu.key] } @@ -634,6 +635,10 @@ < > switch node +
+ l k + switch host +
y n @@ -677,6 +682,10 @@ A open agent picker
+
+ H + open host picker +
Ctrl+C @@ -914,13 +923,22 @@ if (_state.isLoading) hgetConnectionIndicator('loading').classList.add('is-active'); - // 8. Set communication (master / agent) indicator - if (_state.communication.availableAgents.length > 0) { + // 8. Set communication (master / agent / host) indicator + if ( + _state.communication.availableAgents.length > 0 || + _state.communication.availableHosts + ) { hgetConnectionIndicator('communication-target').classList.add( 'is-active' ); - hgetConnectionIndicator('communication-target').textContent = - _state.communication.currentAgent || 'Master'; + + let fullIndicator = _state.communication.currentAgent || 'Master'; + if (_state.communication.currentHost) + fullIndicator = `${fullIndicator} (${_state.communication.currentHost})`; + + hgetConnectionIndicator( + 'communication-target' + ).textContent = fullIndicator; } // 9. Reset mobile controls' visibility @@ -982,6 +1000,9 @@ if (state.communication.currentAgent) copy.Agent = state.communication.currentAgent; + if (state.communication.currentHost) + copy.Host = state.communication.currentHost; + wsSocket.send(JSON.stringify(copy)); }; @@ -1046,7 +1067,7 @@ /** * @typedef {object} Menu * @property {Array} actions - * @property {'menu'|'bulk'|'theme'|'agent'} key + * @property {'menu'|'bulk'|'theme'|'agent'|'host'} key */ /** @@ -1069,7 +1090,7 @@ actions: [], /** - * @type {'menu'|'bulk'|'theme'|'agent'} + * @type {'menu'|'bulk'|'theme'|'agent'|'host'} */ key: null, }, @@ -1080,7 +1101,7 @@ helper: 'default', /** - * @type {"menu"|"bulk"|"prompt"|"message"|"tty"|"help"|"theme"|"agent"} + * @type {"menu"|"bulk"|"prompt"|"message"|"tty"|"help"|"theme"|"agent"|'host'} */ popup: null, @@ -1215,14 +1236,24 @@ communication: { /** - * @type {String} + * @type {string} */ currentAgent: null, /** - * @type {Array} + * @type {Array} */ availableAgents: [], + + /** + * @type {string} + */ + currentHost: null, + + /** + * @type {string} + */ + availableHosts: [], }, _delays: { @@ -1747,6 +1778,15 @@ websocketSend({ action: 'clear' }); }, + /** + * Private - Pick a new host for further communication + * @param {MenuAction} action + */ + _pickHost: function (action) { + state.communication.currentHost = action.Label; + cmdRun(cmds._init); + }, + /** * Public - Quit the app / Quit the current popup * Requires prompt @@ -2382,6 +2422,26 @@ cmdRun(cmds._showPopup, 'menu'); }, + /** + * Public - Show host picker + */ + host: function () { + if (state.communication.availableHosts.length === 0) return; + + state.menu.key = 'host'; + + state.menu.actions = state.communication.availableHosts.map((t) => ({ + RunLocally: true, + RequiresResource: false, + RequiresMenuAction: true, + Label: t, + Command: '_pickHost', + })); + + state.navigation.currentMenuRow = 1; + cmdRun(cmds._showPopup, 'menu'); + }, + /** * Public - Switch to the previous agent for further communication */ @@ -2448,6 +2508,49 @@ cmdRun(cmds._init); }, + + /** + * Public - Switch to the previous host for further communication + */ + previousHost: function () { + if (state.communication.availableHosts.length === 0) return; + + const currentIndex = state.communication.availableHosts.indexOf( + state.communication.currentHost + ); + + if (currentIndex === 0) { + state.communication.currentHost = + state.communication.availableHosts[ + state.communication.availableHosts.length - 1 + ]; + } + // Regular case, switching to the previous agent + else + state.communication.currentHost = + state.communication.availableHosts[currentIndex - 1]; + + cmdRun(cmds._init); + }, + + /** + * Public - Switch to the next host for further communication + */ + nextHost: function () { + if (state.communication.availableHosts.length === 0) return; + + const currentIndex = state.communication.availableHosts.indexOf( + state.communication.currentHost + ); + + if (currentIndex === state.communication.availableHosts.length - 1) { + state.communication.currentHost = state.communication.availableHosts[0]; + } else + state.communication.currentHost = + state.communication.availableHosts[currentIndex + 1]; + + cmdRun(cmds._init); + }, }; // === Variables @@ -2513,6 +2616,10 @@ '<': 'previousAgent', '>': 'nextAgent', + // Hosts + k: 'previousHost', + l: 'nextHost', + // Sub commands q: 'quit', d: 'remove', @@ -2528,6 +2635,7 @@ w: 'browser', h: 'hub', A: 'agent', + H: 'host', G: 'github', T: 'theme', '?': 'help', @@ -2826,11 +2934,22 @@ {} ); - // Update agents' list only on the very first init + // Update agents list only on the very first init if (state.communication.availableAgents.length === 0) state.communication.availableAgents = notification.Content.Agents || []; + // Update hosts list only on the very first init + if (state.communication.availableHosts.length === 0) { + state.communication.availableHosts = notification.Content.Hosts || []; + if (state.communication.availableHosts.length > 0) { + state.communication.currentHost = + state.communication.availableHosts[0]; + + cmdRun(cmds._init); + } + } + state.isLoading = false; cmdRun(cmds._inspectorTabs); break; diff --git a/app/main.go b/app/main.go index 2d3ac23..4c0f448 100644 --- a/app/main.go +++ b/app/main.go @@ -1,6 +1,7 @@ package main import ( + "context" "embed" "errors" "fmt" @@ -51,11 +52,13 @@ func performVerifications() error { } // 2. Ensure Docker socket is reachable - c, err := client.NewClientWithOpts(client.FromEnv) - if err != nil { - return fmt.Errorf("Failed Verification : Access to Docker socket -> %s", err) + if _os.GetEnv("MULTI_HOST_ENABLED") != "TRUE" { + c, err := client.NewClientWithOpts(client.FromEnv) + if err != nil { + return fmt.Errorf("Failed Verification : Access to Docker socket -> %s", err) + } + defer c.Close() } - defer c.Close() // 3. Ensure server port is available l, err := net.Listen("tcp", fmt.Sprintf(":%s", _os.GetEnv("SERVER_PORT"))) @@ -90,19 +93,53 @@ func performVerifications() error { } } + // 7. Ensure docker_hosts file is available when multi-host is enabled + if _os.GetEnv("MULTI_HOST_ENABLED") == "TRUE" { + if _, err := os.Stat("docker_hosts"); errors.Is(err, os.ErrNotExist) { + return fmt.Errorf("Failed Verification : docker_hosts file is missing. Please put it next to the executable") + } + } + + // 8. Ensure every host is reachable if multi-host is enabled, and docker_hosts is well-formatted + if _os.GetEnv("MULTI_HOST_ENABLED") == "TRUE" { + raw, err := os.ReadFile("docker_hosts") + if err != nil { + return fmt.Errorf("Failed Verification : docker_hosts file can't be read -> %s", err) + } + if len(raw) == 0 { + return fmt.Errorf("Failed Verification : docker_hosts file is empty.") + } + + lines := strings.Split(string(raw), "\n") + for _, line := range lines { + if len(line) == 0 { + continue + } + + parts := strings.Split(line, " ") + if len(parts) != 2 { + return fmt.Errorf("Failed Verification : docker_hosts file isn't properly formatted. Line : -> %s", line) + } + + c, err := client.NewClientWithOpts(client.WithHost(parts[1])) + if err != nil { + return fmt.Errorf("Failed Verification : Access to Docker host -> %s", err) + } + + _, err = c.Ping(context.Background()) + if err != nil { + return fmt.Errorf("Failed Verification : Access to Docker host -> %s", err) + } + + c.Close() + } + } + return nil } // Entrypoint func main() { - // Automatically discover the Docker host on the machine - discoveredHost, err := _client.DiscoverDockerHost() - if err != nil { - log.Print(err.Error()) - return - } - os.Setenv("DOCKER_HOST", discoveredHost) - // Load default settings via default.env file (workaround since the file is embed) defaultSettings, _ := godotenv.Unmarshal(defaultEnv) for k, v := range defaultSettings { @@ -112,11 +149,21 @@ func main() { } // Load custom settings via .env file - err = godotenv.Overload(".env") + err := godotenv.Overload(".env") if err != nil { log.Print("No .env file provided, will continue with system env") } + if _os.GetEnv("MULTI_HOST_ENABLED") != "TRUE" { + // Automatically discover the Docker host on the machine + discoveredHost, err := _client.DiscoverDockerHost() + if err != nil { + log.Print(err.Error()) + return + } + os.Setenv("DOCKER_HOST", discoveredHost) + } + // Perform initial verifications if _os.GetEnv("SKIP_VERIFICATIONS") != "TRUE" { // Ensure everything is ready for our app @@ -130,9 +177,38 @@ func main() { } // Set up everything (Melody instance, Docker client, Server settings) - _server := server.Server{ - Melody: melody.New(), - Docker: _client.NewClientWithOpts(client.FromEnv), + var _server server.Server + if _os.GetEnv("MULTI_HOST_ENABLED") != "TRUE" { + _server = server.Server{ + Melody: melody.New(), + Docker: _client.NewClientWithOpts(client.FromEnv), + } + } else { + _server = server.Server{ + Melody: melody.New(), + } + + // Populate server's known hosts when multi-host is enabled + _server.Hosts = make(server.HostsArray, 0) + var firstHost string + + raw, _ := os.ReadFile("docker_hosts") + lines := strings.Split(string(raw), "\n") + for _, line := range lines { + if len(line) == 0 { + continue + } + parts := strings.Split(line, " ") + + _server.Hosts = append(_server.Hosts, []string{parts[0], parts[1]}) + + if len(firstHost) == 0 { + firstHost = parts[0] + } + } + + // Set default Docker client on the first known host + _server.SetHost(firstHost) } _server.Melody.Config.MaxMessageSize = _strconv.ParseInt(_os.GetEnv("SERVER_MAX_READ_SIZE"), 10, 64) diff --git a/app/sample.docker_hosts b/app/sample.docker_hosts new file mode 100644 index 0000000..6609dd2 --- /dev/null +++ b/app/sample.docker_hosts @@ -0,0 +1,3 @@ +Local unix:///var/run/docker.sock +Host-1 tcp://your-domain.tld:your-port +Host-2 tcp://your-ip:your-port diff --git a/app/server/server/hosts.go b/app/server/server/hosts.go new file mode 100644 index 0000000..c1f988f --- /dev/null +++ b/app/server/server/hosts.go @@ -0,0 +1,14 @@ +package server + +// Represent an array of Isaiah hosts ([name, hostname]) +type HostsArray [][]string + +func (hosts HostsArray) ToStrings() []string { + arr := make([]string, 0) + + for _, v := range hosts { + arr = append(arr, v[0]) + } + + return arr +} diff --git a/app/server/server/server.go b/app/server/server/server.go index b54218d..5382fc3 100644 --- a/app/server/server/server.go +++ b/app/server/server/server.go @@ -6,6 +6,7 @@ import ( "io" "slices" "strings" + _client "will-moss/isaiah/server/_internal/client" _io "will-moss/isaiah/server/_internal/io" _os "will-moss/isaiah/server/_internal/os" _session "will-moss/isaiah/server/_internal/session" @@ -22,6 +23,7 @@ type Server struct { Melody *melody.Melody Docker *client.Client Agents AgentsArray + Hosts HostsArray } // Represent a command handler, used only _internally @@ -80,6 +82,7 @@ func (server *Server) runCommand(session _session.GenericSession, command ui.Com volumes := resources.VolumesList(server.Docker) networks := resources.NetworksList(server.Docker) agents := server.Agents.ToStrings() + hosts := server.Hosts.ToStrings() if len(containers) > 0 { columns := strings.Split(_os.GetEnv("COLUMNS_CONTAINERS"), ",") @@ -108,7 +111,7 @@ func (server *Server) runCommand(session _session.GenericSession, command ui.Com server.SendNotification( session, ui.NotificationInit(ui.NotificationParams{ - Content: ui.JSON{"Tabs": tabs, "Agents": agents}, + Content: ui.JSON{"Tabs": tabs, "Agents": agents, "Hosts": hosts}, })) // Command : Agent-only - Clear TTY / Stream @@ -291,6 +294,13 @@ func (server *Server) Handle(session _session.GenericSession, message ...[]byte) return } + // When multi-host is enabled, set the appropriate host before interacting with Docker + if _os.GetEnv("MULTI_HOST_ENABLED") == "TRUE" { + if command.Host != "" { + server.SetHost(command.Host) + } + } + // # - Dispatch the command to the appropriate handler var h handler @@ -327,3 +337,15 @@ func (server *Server) Handle(session _session.GenericSession, message ...[]byte) } } + +func (s *Server) SetHost(name string) { + var correspondingHost []string + for _, v := range s.Hosts { + if v[0] == name { + correspondingHost = v + break + } + } + + s.Docker = _client.NewClientWithOpts(client.WithHost(correspondingHost[1])) +} diff --git a/app/server/ui/command.go b/app/server/ui/command.go index 1342c04..fd69d04 100644 --- a/app/server/ui/command.go +++ b/app/server/ui/command.go @@ -7,6 +7,7 @@ type Command struct { Action string Args map[string]interface{} Agent string + Host string Initiator string Sequence int32 } diff --git a/examples/docker-compose.host.yml b/examples/docker-compose.host.yml new file mode 100644 index 0000000..b6e5c57 --- /dev/null +++ b/examples/docker-compose.host.yml @@ -0,0 +1,11 @@ +version: '3' +services: + isaiah: + image: mosswill/isaiah:latest + restart: unless-stopped + volumes: + - /var/run/docker.sock:/var/run/docker.sock:ro + - my_docker_hosts:/docker_hosts + environment: + AUTHENTICATION_SECRET: "your-very-long-and-mysterious-secret" + MULTI_HOST_ENABLED: "TRUE"