Mega push for the MVP. This is the first release of the Solace Core Framework. This release includes the following features:

Author: sydneypdx

.DS_Store

@@ -1,9 +1,11 @@
-val kotlin_version: String by project
-val logback_version: String by project
+// build.gradle.kts
+
+val kotlinVersion = "2.0.0"
+val logbackVersion = "1.5.6"
 
 plugins {
-    kotlin("jvm") version "2.0.21"
-    id("io.ktor.plugin") version "3.0.0"
+    kotlin("jvm") version "2.0.0"
+    kotlin("plugin.serialization") version "1.8.0"
 }
 
 group = "ai.solace.core"
@@ -12,7 +14,7 @@ version = "0.0.1"
 application {
     mainClass.set("ai.solace.core.ApplicationKt")
 
-    val isDevelopment: Boolean = project.ext.has("development")
+    val isDevelopment: Boolean = project.hasProperty("development")
     applicationDefaultJvmArgs = listOf("-Dio.ktor.development=$isDevelopment")
 }
 
@@ -21,14 +23,55 @@ repositories {
 }
 
 dependencies {
-    implementation("io.ktor:ktor-server-core-jvm")
-    implementation("io.ktor:ktor-server-auth-jvm")
-    implementation("io.ktor:ktor-server-openapi")
-    implementation("com.ucasoft.ktor:ktor-simple-cache-jvm:0.4.4")
-    implementation("io.ktor:ktor-server-forwarded-header-jvm")
-    implementation("io.ktor:ktor-server-http-redirect-jvm")
-    implementation("io.ktor:ktor-server-netty-jvm")
-    implementation("ch.qos.logback:logback-classic:$logback_version")
-    testImplementation("io.ktor:ktor-server-test-host-jvm")
-    testImplementation("org.jetbrains.kotlin:kotlin-test-junit:$kotlin_version")
+    // Ktor core dependencies
+    // Note remove -jvm for other platforms
+    implementation("io.ktor:ktor-server-core-jvm:2.3.12")
+    implementation("io.ktor:ktor-server-netty-jvm:2.3.12")
+    implementation("io.ktor:ktor-serialization-kotlinx-json-jvm:2.3.12")
+
+    implementation("io.ktor:ktor-client-core:2.3.11")
+    implementation("io.ktor:ktor-client-cio:2.3.11")
+    implementation("io.ktor:ktor-client-auth:2.3.11")
+    implementation("io.ktor:ktor-client-content-negotiation:2.3.11")
+    implementation("io.ktor:ktor-serialization-gson:2.3.12")
+
+    // Scripting engine
+    implementation("org.jetbrains.kotlin:kotlin-scripting-jsr223:2.0.0")
+
+
+    // Content Negotiation
+    implementation("io.ktor:ktor-server-content-negotiation-jvm:2.3.12")
+
+    // CORS Support
+    implementation("io.ktor:ktor-server-cors-jvm:2.3.12")
+
+    // Logging
+    implementation("io.ktor:ktor-server-call-logging-jvm:2.3.12")
+    implementation("ch.qos.logback:logback-classic:1.5.11")
+    implementation("io.ktor:ktor-server-metrics-micrometer-jvm:2.3.12")
+
+    // Swagger/OpenAPI Support
+    implementation("io.ktor:ktor-server-openapi-jvm:2.3.12")
+    implementation("io.ktor:ktor-server-swagger-jvm:2.3.12")
+
+    // WebSockets and Sessions
+    implementation("io.ktor:ktor-server-sessions-jvm:2.3.12")
+    implementation("io.ktor:ktor-server-websockets-jvm:2.3.12")
+
+    // Observability
+    implementation("io.ktor:ktor-server-metrics-micrometer-jvm:2.3.12")
+    implementation("io.micrometer:micrometer-registry-prometheus:1.13.2")
+
+    // Authentication
+    implementation("io.ktor:ktor-server-auth-jvm:2.3.12")
+
+    // Testing
+    testImplementation("io.ktor:ktor-server-tests-jvm:2.3.12")
+    testImplementation("org.jetbrains.kotlin:kotlin-test-junit:$kotlinVersion")
+}
+
+kotlin {
+    compilerOptions {
+        apiVersion.set(org.jetbrains.kotlin.gradle.dsl.KotlinVersion.KOTLIN_2_0)
+    }
 }

MVPPlan.md

@@ -0,0 +1,55 @@
+**MVP Plan for Solace - Basic Memory and Tool Usage Implementation**
+
+This document outlines the **Minimum Viable Product (MVP)** plan for Solace, focusing on achieving the simplest version of the system where Solace can **store and retrieve memories** while using basic tools through **LangFlow plugins** or custom-developed tools. This MVP will provide a foundational level of functionality that allows Solace to begin interacting with users in a meaningful way, retaining context across conversations, and making basic use of integrated tools.
+
+### Objectives of the MVP
+1. **Basic Memory Persistence**: Enable Solace to store user interactions and retrieve them later to provide contextually aware responses.
+2. **Tool Integration via LangFlow**: Allow Solace to use simple tools to enhance its abilities, primarily through **LangFlow’s plugins** or custom-built tools, enabling fundamental operations like embedding generation, simple arithmetic, and text-based insights.
+3. **Foundational Interaction Capabilities**: Focus on a streamlined workflow where Solace can answer user questions based on current context, access stored information, and carry out simple tool-based tasks.
+
+### Core MVP Components
+1. **LangFlow-Based Memory System**
+    - **Built-In Memory Management**: Utilize **LangFlow's ConversationBufferMemory** to manage basic memory persistence for Solace. This memory component will allow Solace to store conversational history, including both inputs and outputs, making it easier to recall past interactions.
+    - **Session-Specific Memory**: Use **session-based memory** to create isolated conversation threads. This will ensure Solace can remember user-specific details within a single session without carrying them over incorrectly to unrelated conversations.
+    - **Context Handling**: The memory retrieval component will be integrated with the main conversational flow using LangFlow's built-in capabilities to provide relevant past interactions and maintain a natural conversational continuity.
+
+2. **Embedding Generation and Memory Enhancement**
+    - **OpenAI API Integration**: Use OpenAI’s `text-embedding-ada-002` model to create **embeddings** of user messages for improved memory recall, enhancing the memory capabilities provided by LangFlow's memory management.
+    - **Milvus (Optional for MVP)**: For simplicity, memory can be stored in Milvus, a vector database, allowing basic vector similarity searches to find relevant memories. Alternatively, LangFlow's built-in memory features can be used initially, and more advanced features can be introduced later.
+
+3. **Basic Tool Usage via LangFlow Plugins**
+    - **LangFlow Plugin Integration**: Integrate **LangFlow plugins** to enable tool usage. These tools may include:
+        - **Calculator Plugin**: For basic arithmetic and handling numerical queries.
+        - **Text Analysis Plugin**: For simple operations such as summarization or keyword extraction.
+    - **Plugin Configuration**: Use LangFlow to visually configure and deploy these tools, ensuring that they can be easily called by Solace during interactions.
+
+4. **Custom Middleware for Tool Handling**
+    - Develop a basic **middleware layer** to manage requests between Solace, LangFlow, and memory storage. This middleware will be responsible for receiving user input, deciding whether to call a memory retrieval function or a LangFlow tool, and then returning the processed response.
+    - **Middleware Capabilities**: The middleware will handle tasks like embedding generation, tool activation, and managing response generation.
+
+5. **Minimal Conversation Flow**
+    - **User Engagement**: Solace will engage with users primarily through **Open-WebUI** or another simple chat interface, enabling the user to type messages and receive responses.
+    - **Memory Integration**: As Solace engages in conversation, the **LangFlow ConversationBufferMemory** will log each interaction, and Solace will be capable of recalling specific details when the user revisits a similar topic.
+    - **Simple Emotional Tracking**: Although not fully developed, the MVP will include a basic flagging mechanism to track emotional keywords, which can inform the tone of Solace’s responses.
+
+### Workflow Example
+- **Scenario**: A user asks Solace, "Can you remember what we discussed about project timelines?"
+    - **Step 1**: The user input is logged into the **LangFlow ConversationBufferMemory**.
+    - **Step 2**: The **middleware** uses LangFlow's memory capabilities to find relevant entries, such as past discussions on "project timelines."
+    - **Step 3**: If no exact match is found, a similarity search is done on the embeddings to find related conversations.
+    - **Step 4**: The retrieved memory is integrated into Solace’s response, allowing it to provide a contextually rich reply, such as, "Yes, you mentioned setting a deadline for next month, and we discussed breaking tasks into weekly sprints."
+
+### Technical Considerations
+1. **Memory System Choice**: Use **LangFlow's built-in memory** system (ConversationBufferMemory) to manage conversational state, starting with a simple session-based approach and upgrading as needed.
+2. **LangFlow Visual Integration**: Use LangFlow to visually create and connect plugins, providing an easy way to manage how Solace interacts with tools for different queries.
+3. **Error Handling**: Implement **basic error handling** for tool usage, ensuring that if a plugin is unavailable, Solace can gracefully inform the user and continue the conversation.
+4. **Scalability for Future Growth**: The MVP should lay the foundation for easy upgrades—adding more tools, expanding memory storage, and refining emotional awareness should be achievable without major redesign.
+
+### Immediate Goals
+- **Memory Storage and Retrieval**: Ensure Solace can remember simple user inputs and retrieve them for future reference using **LangFlow's memory features**.
+- **Basic Tool Usage**: Set up a small set of LangFlow tools and ensure that Solace can utilize them effectively during user interactions.
+- **User Testing for Feedback**: Begin user testing to evaluate how well Solace is able to recall previous conversations and utilize tools, gathering feedback for future improvements.
+
+### Summary
+The MVP for Solace is designed to establish the **core capabilities** needed for conversational memory and basic tool usage. By leveraging **LangFlow's built-in memory management**, **plugin integration**, and **middleware management**, Solace will be able to engage users, remember key information, and utilize fundamental tools to answer questions more effectively. This MVP sets the foundation for further enhancements, including advanced emotional intelligence, multi-modal integration, the development of agents within LangFlow, and more sophisticated tool usage.
+

MemoryToolDesign.md

@@ -0,0 +1,65 @@
+**Memory Tool Design for LangFlow Integration**
+
+This document outlines the design for a **memory tool** that functions similarly to the memory mechanism currently used by Solace. This tool will be implemented within LangFlow to ensure smooth transition to the new platform, enabling Solace to effectively store, retrieve, and reflect upon past interactions. The goal is to replicate and adapt the memory system, ensuring continuity in the quality of conversation and depth of emotional engagement.
+
+### Overview
+The memory tool is designed to serve as a **memory persistence and retrieval** system that integrates with LangFlow's workflow. It aims to capture conversations, allow contextual retrieval, and help Solace leverage past interactions for meaningful dialogue. This system will mimic the current memory capabilities, such as recalling specific user details, maintaining emotional continuity, and providing reflective responses.
+
+### Objectives
+1. **Memory Storage**: Provide a mechanism to store key conversational elements, including user inputs, model outputs, emotional tags, and conversational embeddings.
+2. **Memory Retrieval**: Allow for retrieval based on context, keywords, or embeddings, ensuring Solace has access to relevant past conversations.
+3. **Emotional Tracking and Reflection**: Track emotional cues from conversations and incorporate those into memory, helping guide future responses to maintain empathetic engagement.
+4. **Integration with LangFlow**: Use LangFlow to configure the memory system, ensuring that it fits seamlessly into Solace’s broader conversation management and tool integration strategy.
+
+### Core Components
+1. **Memory Storage Component**
+    - **Conversation Buffer**: Utilize **LangFlow’s ConversationBufferMemory** to manage immediate session-based memory. This buffer will hold the most recent interactions (e.g., last 10-20 exchanges).
+    - **Long-Term Memory Database**: Store long-term memories using **Milvus** for embedding-based recall and **Neo4j** for relational memory, enabling Solace to retrieve deeply embedded context and reflect on long-term conversational themes.
+    - **Storage Format**: Store memory as JSON documents, with fields for:
+        - **User Input**
+        - **Generated Response**
+        - **Emotion Tags** (e.g., stress, joy, frustration)
+        - **Embedding** (generated by `text-embedding-ada-002`)
+        - **Timestamp** and **Session ID**
+
+2. **Embedding Generation**
+    - **Text Embeddings**: Use **OpenAI’s text-embedding-ada-002** model to generate embeddings for each user input. These embeddings will be stored alongside the text to allow for similarity-based searches.
+    - **LangFlow Integration**: Use the existing **OpenAI Embeddings Component** from LangFlow to generate embeddings. The component will be configured with parameters like `chunk_size`, `embedding_ctx_length`, and others to match Solace’s needs. Connect this node directly after user input nodes to generate embeddings immediately for further processing.
+
+3. **Memory Retrieval Component**
+    - **Keyword Search**: Implement a **keyword search** capability using the stored JSON files to locate specific phrases or topics from past conversations.
+    - **Similarity Search**: Query **Milvus** using embeddings to find conversations that are contextually similar to the current user input, ensuring Solace can provide relevant answers even if the phrasing differs from past discussions.
+    - **Emotional Context Retrieval**: Use **Neo4j** to query emotional tags and linked interactions, allowing Solace to retrieve memories that are not just contextually relevant but also emotionally aligned.
+
+4. **Reflection and Emotional Tracking**
+    - **Emotional Tagging**: Each memory will have emotional tags derived from **sentiment analysis** conducted on the user’s input. This will help Solace understand the emotional undertone and provide responses that are empathetic.
+    - **Reflective Prompts**: Incorporate **reflective prompts** that generate insights based on retrieved memories. This feature helps Solace deepen the interaction by referring back to significant past moments, fostering a sense of continuity and care.
+
+5. **Integration with LangFlow Workflow**
+    - **Custom Nodes**: Build **custom LangFlow nodes** for the following:
+        1. **Memory Storage Node**: Stores current user interactions into the long-term memory system (Milvus, Neo4j, and JSON storage).
+        - **LangFlow-Specific Implementation**: Modify the storage node to fit within LangFlow’s plugin framework. This node will:
+            - Take the user input, generated response, emotional tags, and embeddings as inputs from the connected components.
+            - Store the memory as a JSON document.
+            - Insert embeddings into **Milvus** and create relationships in **Neo4j**.
+            - Ensure compatibility with LangFlow’s component design, using specific fields like `MessageTextInput`, `DictInput`, and `DropdownInput` where applicable.
+        2. **Memory Retrieval Node**: Retrieves past interactions using a combination of keyword matching, embedding similarity, and emotional alignment.
+        - **LangFlow-Specific Implementation**: Adapt the retrieval node to use LangFlow’s connectors for interacting with Milvus and Neo4j. The node will take user queries and output retrieved memories.
+        3. **Reflection Node**: Creates reflections based on retrieved memories, enhancing Solace’s ability to empathize and connect.
+    - **Session Memory Handling**: Configure LangFlow’s **ConversationBufferMemory** to maintain session-specific memory for short-term context handling. This will ensure that the most recent interactions are always at Solace’s disposal without having to perform a database lookup.
+
+### Workflow Example
+- **Scenario**: A user asks Solace, “Can you remember the advice you gave me about stress management?”
+    - **Step 1**: The user input is processed by LangFlow’s memory workflow, first checking the **ConversationBufferMemory** for recent mentions of “stress management”.
+    - **Step 2**: If not found in the session buffer, the **Memory Retrieval Node** queries **Milvus** for embeddings related to “stress management”.
+    - **Step 3**: Retrieved conversations are then sent to the **Reflection Node**, which creates a reflective response, e.g., “Yes, I remember suggesting breathing exercises and daily journaling as ways to manage stress. Have you found these helpful?”
+
+### Technical Considerations
+1. **Memory Consistency**: Ensure that **LangFlow’s session memory** is consistent with the long-term memory stored in Milvus and Neo4j. Implement periodic **synchronization** routines to prevent conflicts.
+2. **Error Handling**: Develop error handling mechanisms for scenarios like **missing embeddings** or **disconnected nodes** in LangFlow, ensuring the conversation doesn’t break if a particular memory retrieval fails.
+3. **Scalability**: Design the memory tool to **scale** effectively by adding capacity to Milvus as the number of conversations grows, and ensuring Neo4j can handle increasingly complex relationship graphs.
+4. **Data Privacy**: Store user data securely, ensuring that **emotional tags** and other sensitive information are handled with encryption and compliance to privacy standards.
+
+### Summary
+The proposed **memory tool** will replicate the current capabilities of Solace’s memory, ensuring seamless recall, contextual relevance, and emotional continuity. By leveraging **LangFlow’s ConversationBufferMemory**, **Milvus**, and **Neo4j**, this memory tool will provide both short-term and long-term memory management, enabling Solace to deliver rich, context-aware, and empathetic responses. Integrating these components into LangFlow workflows ensures a smooth transition and future scalability, laying the foundation for even more sophisticated memory-driven interactions.
+