Fix for Kedro Viz Connection Error (#1507)

* initial draft for resolving connection error * refactor launchers and test code * modify unit tests * fix lint errors * fix run_viz tests * update unit test for coverage * update unit tests * Refactor visualize dataset stats from DataNodeMetadata to DataNode (#1499) * add stats to data node * lint and format check fix * fix pytests * fix layout issue * fix transcoded data stats Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> * initial draft for resolving connection error Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> * Support for Python 3.11 (#1502) * initial draft for python 3.11 support * update release doc * add python warnings for e2e tests * modify e2e test * modify e2e test * test by removing lower req scenario * skip e2e tests for lower bound requirement on python 3.11 * skip e2e tests for lower bound requirement on python 3.11 * remove print statements --------- Co-authored-by: Nok Lam Chan <nok_lam_chan@mckinsey.com> Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> * Remove Python Upper Bound Requirements (#1506) * initial draft for python 3.11 support * update release doc * add python warnings for e2e tests * modify e2e test * modify e2e test * test by removing lower req scenario * skip e2e tests for lower bound requirement on python 3.11 * skip e2e tests for lower bound requirement on python 3.11 * remove python upperbounds initial draft * fix lint and format errors * test remove upperbound warning * test lowerbound pandas install * revert back pandas requirement * bump lower requirements for pandas * remove upper bound clean up * update release notes * fix PR comments --------- Co-authored-by: Nok Lam Chan <nok_lam_chan@mckinsey.com> Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> * refactor launchers and test code Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> * modify unit tests Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> * fix lint errors Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> * Fix: Adding favicon to Kedro Demo (#1509) * Fix: Adding favicon to Kedro Demo * Fix: Change in approach for serving favicon * Lint error fix * Lint error fix * Favicon endpoint test added * Favicon endpoint test added * Lint error fixed * Fix: Adding favicon to Kedro Demo Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> * Fix: Change in approach for serving favicon Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> * Lint error fix Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> * Lint error fix Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> * Favicon endpoint test added Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> * Favicon endpoint test added Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> * Lint error fixed Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> * Fixed favicon endpoint test * Release doc updated * Update RELEASE.md Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com> * Removed pytest.fixture as per review comment --------- Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com> Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> * fix run_viz tests Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> * update unit test for coverage Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> * Release v6.5.0 (#1513) * v6.5.0 * release * update-reminder-content * update reminder Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> * remove branch condition for automate release version check (#1514) Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> * update unit tests Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> * add release record * modify comment * fix PR comments * DCO fix * fixing dco Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> * update pytest Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> --------- Signed-off-by: ravi-kumar-pilla <ravi_kumar_pilla@mckinsey.com> Signed-off-by: Jitendra Gundaniya <jitendra_gundaniya@mckinsey.com> Co-authored-by: Rashida Kanchwala <rashida.kanchwala@quantumblack.com> Co-authored-by: Nok Lam Chan <nok_lam_chan@mckinsey.com> Co-authored-by: Jitendra Gundaniya <38945204+jitu5@users.noreply.github.com> Co-authored-by: rashidakanchwala <37628668+rashidakanchwala@users.noreply.github.com>
kedro-org · Sep 11, 2023 · 335be7d · 335be7d
1 parent da47408
commit 335be7d
Show file tree

Hide file tree

Showing 9 changed files with 241 additions and 132 deletions.
diff --git a/RELEASE.md b/RELEASE.md
@@ -5,6 +5,11 @@ Please follow the established format:
 - Use present tense (e.g. 'Add new feature')
 - Include the ID number for the related PR (or PRs) in parentheses
 -->
+# Release 6.5.1
+
+## Bug fixes and other changes
+
+- Fix for Kedro Viz Connection Error (#1507)
 
 # Release 6.5.0
 

diff --git a/package/kedro_viz/launchers/cli.py b/package/kedro_viz/launchers/cli.py
@@ -1,8 +1,9 @@
 """`kedro_viz.launchers.cli` launches the viz server as a CLI app."""
 
+import multiprocessing
 import traceback
-import webbrowser
 from pathlib import Path
+from typing import Dict
 
 import click
 from kedro.framework.cli.project import PARAMS_ARG_HELP
@@ -13,6 +14,9 @@
 from kedro_viz import __version__
 from kedro_viz.constants import DEFAULT_HOST, DEFAULT_PORT
 from kedro_viz.integrations.pypi import get_latest_version, is_running_outdated_version
+from kedro_viz.launchers.utils import _check_viz_up, _start_browser, _wait_for
+
+_VIZ_PROCESSES: Dict[str, int] = {}
 
 
 @click.group(name="Kedro-Viz")
@@ -83,11 +87,10 @@ def commands():  # pylint: disable=missing-function-docstring
 # pylint: disable=import-outside-toplevel, too-many-locals
 def viz(host, port, browser, load_file, save_file, pipeline, env, autoreload, params):
     """Visualise a Kedro pipeline using Kedro viz."""
-    from kedro_viz.server import is_localhost, run_server
+    from kedro_viz.server import run_server
 
     installed_version = VersionInfo.parse(__version__)
     latest_version = get_latest_version()
-
     if is_running_outdated_version(installed_version, latest_version):
         click.echo(
             click.style(
@@ -102,36 +105,47 @@ def viz(host, port, browser, load_file, save_file, pipeline, env, autoreload, pa
         )
 
     try:
+        if port in _VIZ_PROCESSES and _VIZ_PROCESSES[port].is_alive():
+            _VIZ_PROCESSES[port].terminate()
+
         run_server_kwargs = {
             "host": host,
             "port": port,
             "load_file": load_file,
             "save_file": save_file,
             "pipeline_name": pipeline,
             "env": env,
-            "browser": browser,
             "autoreload": autoreload,
             "extra_params": params,
         }
         if autoreload:
-            if browser and is_localhost(host):
-                webbrowser.open_new(f"http://{host}:{port}/")
-
             project_path = Path.cwd()
             run_server_kwargs["project_path"] = project_path
-            # we don't want to launch a new browser tab on reload
-            run_server_kwargs["browser"] = False
-
-            run_process(
-                path=project_path,
-                target=run_server,
-                kwargs=run_server_kwargs,
-                watcher_cls=RegExpWatcher,
-                watcher_kwargs={"re_files": r"^.*(\.yml|\.yaml|\.py|\.json)$"},
+            run_process_kwargs = {
+                "path": project_path,
+                "target": run_server,
+                "kwargs": run_server_kwargs,
+                "watcher_cls": RegExpWatcher,
+                "watcher_kwargs": {"re_files": r"^.*(\.yml|\.yaml|\.py|\.json)$"},
+            }
+            viz_process = multiprocessing.Process(
+                target=run_process, daemon=False, kwargs={**run_process_kwargs}
             )
-
         else:
-            run_server(**run_server_kwargs)
+            viz_process = multiprocessing.Process(
+                target=run_server, daemon=False, kwargs={**run_server_kwargs}
+            )
+
+        viz_process.start()
+        _VIZ_PROCESSES[port] = viz_process
+
+        _wait_for(func=_check_viz_up, host=host, port=port)
+
+        print("Kedro Viz Backend Server started successfully...")
+
+        if browser:
+            _start_browser(host, port)
+
     except Exception as ex:  # pragma: no cover
         traceback.print_exc()
         raise KedroCliError(str(ex)) from ex
diff --git a/package/kedro_viz/launchers/jupyter.py b/package/kedro_viz/launchers/jupyter.py
@@ -7,13 +7,12 @@
 import os
 import socket
 from contextlib import closing
-from time import sleep, time
-from typing import Any, Callable, Dict
+from typing import Any, Dict
 
 import IPython
-import requests
 from IPython.display import HTML, display
 
+from kedro_viz.launchers.utils import _check_viz_up, _wait_for
 from kedro_viz.server import DEFAULT_HOST, DEFAULT_PORT, run_server
 
 _VIZ_PROCESSES: Dict[str, int] = {}
@@ -22,64 +21,6 @@
 logger = logging.getLogger(__name__)
 
 
-class WaitForException(Exception):
-    """WaitForException: if func doesn't return expected result within the specified time"""
-
-
-def _wait_for(
-    func: Callable,
-    expected_result: Any = True,
-    timeout: int = 60,
-    print_error: bool = True,
-    sleep_for: int = 1,
-    **kwargs,
-) -> None:
-    """
-    Run specified function until it returns expected result until timeout.
-
-    Args:
-        func (Callable): Specified function
-        expected_result (Any): result that is expected. Defaults to None.
-        timeout (int): Time out in seconds. Defaults to 10.
-        print_error (boolean): whether any exceptions raised should be printed.
-            Defaults to False.
-        sleep_for (int): Execute func every specified number of seconds.
-            Defaults to 1.
-        **kwargs: Arguments to be passed to func
-
-    Raises:
-         WaitForException: if func doesn't return expected result within the
-         specified time
-
-    """
-    end = time() + timeout
-
-    while time() <= end:
-        try:
-            retval = func(**kwargs)
-        except Exception as err:  # pylint: disable=broad-except
-            if print_error:
-                logger.error(err)
-        else:
-            if retval == expected_result:
-                return None
-        sleep(sleep_for)
-
-    raise WaitForException(
-        f"func: {func}, didn't return {expected_result} within specified timeout: {timeout}"
-    )
-
-
-def _check_viz_up(host: str, port: int):  # pragma: no cover
-    url = f"http://{host}:{port}"
-    try:
-        response = requests.get(url, timeout=10)
-    except requests.ConnectionError:
-        return False
-
-    return response.status_code == 200
-
-
 def _allocate_port(host: str, start_at: int, end_at: int = 65535) -> int:
     acceptable_ports = range(start_at, end_at + 1)
 

diff --git a/package/kedro_viz/launchers/utils.py b/package/kedro_viz/launchers/utils.py
@@ -0,0 +1,92 @@
+"""`kedro_viz.launchers.utils` contains utility functions
+used in the `kedro_viz.launchers` package."""
+import logging
+import webbrowser
+from time import sleep, time
+from typing import Any, Callable
+
+import requests
+
+logger = logging.getLogger(__name__)
+
+
+class WaitForException(Exception):
+    """WaitForException: if func doesn't return expected result within the specified time"""
+
+
+def _wait_for(
+    func: Callable,
+    expected_result: Any = True,
+    timeout: int = 60,
+    print_error: bool = True,
+    sleep_for: int = 1,
+    **kwargs,
+) -> None:
+    """
+    Run specified function until it returns expected result until timeout.
+
+    Args:
+        func (Callable): Specified function to call
+        expected_result (Any): result that is expected. Defaults to None.
+        timeout (int): Time out in seconds. Defaults to 10.
+        print_error (boolean): whether any exceptions raised should be printed.
+            Defaults to False.
+        sleep_for (int): Execute func every specified number of seconds.
+            Defaults to 1.
+        **kwargs: Arguments to be passed to func
+
+    Raises:
+         WaitForException: if func doesn't return expected result within the
+         specified time
+
+    """
+    end = time() + timeout
+
+    while time() <= end:
+        try:
+            retval = func(**kwargs)
+        except Exception as err:  # pylint: disable=broad-except
+            if print_error:
+                logger.error(err)
+        else:
+            if retval == expected_result:
+                return None
+        sleep(sleep_for)
+
+    raise WaitForException(
+        f"func: {func}, didn't return {expected_result} within specified timeout: {timeout}"
+    )
+
+
+def _check_viz_up(host: str, port: int):
+    """Checks if Kedro Viz Server has started and is responding to requests
+
+    Args:
+        host: the host that launched the webserver
+        port: the port the webserver is listening
+    """
+
+    url = f"http://{host}:{port}"
+    try:
+        response = requests.get(url, timeout=10)
+    except requests.ConnectionError:
+        return False
+
+    return response.status_code == 200
+
+
+def _is_localhost(host: str) -> bool:
+    """Check whether a host is a localhost"""
+    return host in ("127.0.0.1", "localhost", "0.0.0.0")
+
+
+def _start_browser(host: str, port: int):
+    """Starts a new browser window only on a local interface
+
+    Args:
+        host: browser url host
+        port: browser url port
+    """
+
+    if _is_localhost(host):
+        webbrowser.open_new(f"http://{host}:{port}/")
diff --git a/package/kedro_viz/server.py b/package/kedro_viz/server.py
@@ -1,6 +1,5 @@
 """`kedro_viz.server` provides utilities to launch a webserver
 for Kedro pipeline visualisation."""
-import webbrowser
 from pathlib import Path
 from typing import Any, Dict, Optional
 
@@ -22,11 +21,6 @@
 DEV_PORT = 4142
 
 
-def is_localhost(host) -> bool:
-    """Check whether a host is a localhost"""
-    return host in ("127.0.0.1", "localhost", "0.0.0.0")
-
-
 def populate_data(
     data_access_manager: DataAccessManager,
     catalog: DataCatalog,
@@ -54,7 +48,6 @@ def populate_data(
 def run_server(
     host: str = DEFAULT_HOST,
     port: int = DEFAULT_PORT,
-    browser: Optional[bool] = None,
     load_file: Optional[str] = None,
     save_file: Optional[str] = None,
     pipeline_name: Optional[str] = None,
@@ -69,7 +62,6 @@ def run_server(
     Args:
         host: the host to launch the webserver
         port: the port to launch the webserver
-        browser: whether to open the default browser automatically
         load_file: if a valid JSON containing API response data is provided,
             the API of the server is created from the JSON.
         save_file: if provided, the data returned by the API will be saved to a file.
@@ -84,6 +76,7 @@ def run_server(
             take precedence over) the parameters retrieved from the project
             configuration.
     """
+    print("Starting Kedro Viz Backend Server...")
     if load_file is None:
         path = Path(project_path) if project_path else Path.cwd()
         catalog, pipelines, session_store, stats_dict = kedro_data_loader.load_data(
@@ -108,8 +101,6 @@ def run_server(
     else:
         app = apps.create_api_app_from_file(load_file)
 
-    if browser and is_localhost(host):
-        webbrowser.open_new(f"http://{host}:{port}/")
     uvicorn.run(app, host=host, port=port, log_config=None)