AppNexus - The API Layer for contentcuration

contentcuration/contentcuration/tests/appnexus/test_base.py

@@ -0,0 +1,44 @@
+from contentcuration.utils.appnexus.base import Backend, Adapter
+
+class MockBackend(Backend):
+    _instance = None
+
+    def connect(self) -> None:
+        return super().connect()
+
+    def make_request(self, url: str, params=None):
+        return super().make_request(url, params)
+
+    def request(self) -> None:
+        return super().request()
+
+    def response(self) -> None:
+        return super().response()
+
+    @classmethod
+    def _create_instance(cls) -> 'MockBackend':
+        return cls()
+
+
+class MockAdapter(Adapter):
+    def mockoperation(self):
+        pass
+
+
+def test_backend_singleton():
+    b1, b2 = MockBackend.get_instance(), MockBackend.get_instance()
+    assert id(b1) ==  id(b2)
+
+def test_adapter_creation():
+    a = MockAdapter()
+    assert isinstance(a, Adapter)
+
+def test_adapter_backend_default():
+    b = MockBackend()
+    adapter = Adapter(backend=b)
+    assert isinstance(adapter.backend, Backend)
+
+def test_adapter_backend_custom():
+    b = MockBackend()
+    a = Adapter(backend=b)
+    assert a.backend is b

contentcuration/contentcuration/utils/appnexus/APILayer.md

@@ -0,0 +1,161 @@
+## API Layer Documentation
+
+### Overview
+
+Within the `contentcuration` app in Studio, we want to build an API layer that acts as a communication bridge with different backends like Docker Images, Google Cloud Platform's Vertex AI, and VM instances, cloud storage services, etc. The goal is to make sure this API layer can work with these backends, regardless of where or how they do the job. As long as the input and output formats stay the same, this setup provides flexibility in choosing and using backend resources.
+
+### Description and outcomes
+
+The stand-alone deployed backend service(s) will not have direct access to `contentcuration` models or the database for that matter, so this API layer facilitates access to these resources by receiving and returning a standardized requests and responses, irrespective of the backend interacted with.
+
+#### The Architecture
+
+<img width="1178" alt="Screenshot 2023-09-11 at 14 50 06" src="https://github.com/learningequality/studio/assets/5203639/05f64be8-af39-466b-9e13-00426d07b7ff">
+
+The key components of this architecture are as follows:
+
+#### 1. Creating the Backend Interface
+
+The Backend class serves as an abstract interface that outlines the operations all backends must support. It implements the Singleton pattern to ensure that only one instance of each backend type can exist. The methods defined by the Backend class are:
+
+```python
+ABSTRACT CLASS Backend:
+	_instance = None # Private variable to hold the instance
+
+	ABSTRACT METHOD connect()
+		# Provides blue print to connect
+		pass
+
+	ABSTRACT METHOD make_request(params)
+		# provide blue print to make request
+		pass
+
+    ABSTRACT METHOD request(params)
+		# provide blue print for the request object
+		pass
+
+	ABSTRACT METHOD response(params)
+		# provides blue print for the response object
+		pass
+
+	CLASS METHOD get_instance(cls)
+		IF cls._instance is None:
+			cls._instance = cls._create_instance()
+		return cls._instance
+
+	CLASS METHOD _create_instance(cls)
+		raise NotImplementedError # concrete class must implement
+```
+
+Different backends can now be created by implementing the base `Backend` class:
+
+```python
+# Implement CONCRETE CLASS using ABSTRACT Backend class
+CLASS GCS IMPLEMENTS Backend:
+	METHOD make_request(params):
+		# make request to Google Cloud Storage services
+
+	METHOD connect(params):
+		# Implement the connect method for GCS
+
+	CLASS METHOD _create_instance(cls)
+		# initialize a GCS Backend instance
+
+CLASS ML IMPLEMENTS Backend:
+	METHOD make_request(params):
+		# make request to DeepLearning models hosted as service
+
+	METHOD connect(params):
+		# Implement the connect method for hosted ML service
+
+	CLASS METHOD _create_instance(cls)
+		# initialize a ML Backend instance
+
+CLASS OtherBackend IMPLEMENTS Backend:
+	...
+	[you get the idea]
+```
+
+To create an instance  of a backend, using  the `ML` class as an example, use the `get_instance()` method:
+
+```python
+>>> backend = ML.get_instance()
+```
+
+To centralize the creation of `Backend` instances based on specific Django settings(e.g. dev vs. production environments),  create `BackendFactory` class. This should follow the Factory Design Pattern.
+
+```python
+# Factory to instantiate the Backend based on Django Settings
+CLASS BackendFactory:
+	METHOD create_backend(self, backend=None) -> Backend
+		IF backend:
+			return backend
+		ELSE:
+			# Create an Adapter instance based on Django settings
+			IF DjangoSettings is 'SomeSetting':
+				backend = GCS.get_instance() # Use of Singleton pattern
+			ELSE IF DjangoSettings is 'AnotherSetting':
+				backend = ML.get_instance()
+			ELSE
+				RAISE ValueError
+		# Return the created Backend instance
+		RETURN backend
+```
+The `BackendFactory`'s `create_backend` method optionally allows a `Backend` instance to be injected into the factory instead of relying solely on Django settings. This is particularly useful if we want to explicitly specify the backend to use.
+
+### Creating Adapter that accepts any Backend
+
+The **`Adapter`** class can be initialized with a `Backend` instance(optional) which provides a `make_request` method that forwards requests to the chosen `Backend`, while adhering to its specific `request` and `response` formats.
+
+```python
+CLASS Adapter:
+
+	METHOD __init__(self, backend(Optional) defaults None)
+		# Initialize the Backend with BackendFactory
+		backend_factory = BackendFactory()
+		SET backend = backend_factory.create_backend(backend)
+
+	METHOD request(self):
+		# something
+		return self.backend.request()
+
+	METHOD response(self):
+		# something
+		return self.backend.response()
+```
+
+With this `Adapter` class in place, we can create Adapter that are able interact with any backend we need.
+
+```python
+CLASS Recommendation INHERITS ADAPTER:
+	METHOD generateEmbeddings(self, params) -> Boolean
+		# [ Implementation ]
+
+	METHOD getRecommendation(self, params) -> Array
+		# [ Implementation ]
+
+CLASS Transcription INHERITS ADAPTER:
+	METHOD generateCaption(self, params) -> Array
+		# [ Implementation ]
+
+CLASS OtherAdapter INHERITS ADAPTER:
+	METHOD someOperation(self, params) -> Any
+		# Operation that any backend wants
+```
+
+Below is a sample use case, using the `ML` backend as an example:
+
+```python
+>>> backend = ML.get_instance()
+>>> adapter =  Transcription(backend)
+```
+
+To access specific methods within the adapter:
+
+```python
+>>> adapter.generateCaption(...)
+```
+
+### Resources
+
+[OOP Design patterns](https://refactoring.guru/design-patterns/catalog)

contentcuration/contentcuration/utils/appnexus/__init__.py

@@ -0,0 +1,2 @@
+from .adapters import *
+from .backends import *

contentcuration/contentcuration/utils/appnexus/adapters.py

@@ -0,0 +1 @@
+# A file to implement adapters like Recommendation, Transcription etc

contentcuration/contentcuration/utils/appnexus/backends.py

@@ -0,0 +1 @@
+# Implementation of ML, GCS Backend etc.

contentcuration/contentcuration/utils/appnexus/base.py

@@ -0,0 +1,81 @@
+from abc import ABC, abstractmethod
+from builtins import NotImplementedError
+from typing import Optional, Union, Dict
+
+
+class Backend(ABC):
+    """ An abstract base class for backend interfaces that also implements the singleton pattern """
+    _instance = None
+
+    def __new__(class_, *args, **kwargs):
+        if not isinstance(class_._instance, class_):
+            class_._instance = object.__new__(class_, *args, **kwargs)
+        return class_._instance
+
+    @abstractmethod
+    def connect(self) -> None:
+        """ Establishes a connection to the backend service. """
+        pass
+
+    @abstractmethod
+    def make_request(self, url: str, params=None) -> Union[bytes, str, Dict]:
+        """ Makes an HTTP request to a given URL using the specified method. """
+        pass
+
+    @abstractmethod
+    def request(self) -> None:
+        """ Blueprint for the request object. """
+        pass
+
+    @abstractmethod
+    def response(self) -> None:
+        """ Blueprint for the response object. """
+        pass
+
+    @classmethod
+    def get_instance(cls) -> 'Backend':
+        """ Returns existing instance, if not then create one. """
+        return cls._instance if cls._instance else cls._create_instance()
+
+    @classmethod
+    def _create_instance(cls) -> 'Backend':
+        """ Returns the instance after creating it. """
+        raise NotImplementedError("Subclasses should implement the creation of instance")
+
+
+class BackendFactory:
+    def create_backend(self, backend: Optional[Backend] = None) -> Backend:
+        """
+        Create a Backend instance based on Django or manual settings.
+
+        Args:
+            backend (Optional[Backend], optional): 
+            An optional pre-existing Backend instance.
+
+        Returns:
+            Backend: A Backend instance.
+        """
+        return backend or self._create_backend_from_settings()
+
+    def _create_backend_from_settings(self) -> Backend:
+        # TODO: use Django settings to create backend.
+        pass
+
+
+class Adapter:
+    """
+    Base class for adapters that interact with a backend interface.
+
+    This class should be inherited by adapter classes that facilitate
+    interaction with different backend implementation.
+    """
+    def __init__(self, backend: Optional[Backend] = None) -> None:
+        self.backend = BackendFactory().create_backend(backend=backend)
+
+    def request(self):
+        """ Forward the request to the chosen Backend """
+        return self.backend.request()
+
+    def response(self):
+        """ Forward the response to the chosen Backend """
+        return self.backend.response()