llm web command - launches a web server #4

simonw · 2023-04-02T01:10:50Z

A command that launches a web server to allow you to both browse your logs and interact with APIs through your browser.

simonw · 2023-04-02T01:11:00Z

I'm tempted to add Datasette as a dependency for this.

simonw · 2023-04-02T01:14:32Z

Maybe this becomes a two-way thing: it could be a Datasette plugin (which adds datasette llm as a command, and implements the full UI in Datasette) but it could also be a thing where if you install llm and run llm web you get a Datasette interface running that plugin.

simonw · 2023-04-02T04:09:33Z

Prototype:

import asyncio
from datasette import hookimpl, Response
import openai



CHAT = """
<!DOCTYPE html>
<html>
<head>
    <title>WebSocket Client</title>
</head>
<body>
    <h1>WebSocket Client</h1>
    <textarea id="message" rows="4" cols="50"></textarea><br>
    <button onclick="sendMessage()">Send Message</button>
    <div id="log" style="margin-top: 1em; white-space: pre-wrap;"></div>

    <script>
        const ws = new WebSocket(`ws://${location.host}/ws`);

        ws.onmessage = function(event) {
            console.log(event);
            const log = document.getElementById('log');
            log.textContent += event.data;
        };

        function sendMessage() {
            const message = document.getElementById('message').value;
            console.log({message, ws});
            ws.send(message);
        }
    </script>
</body>
</html>
""".strip()


async def websocket_application(scope, receive, send):
    from .cli import get_openai_api_key
    openai.api_key = get_openai_api_key()
    if scope["type"] != "websocket":
        return Response.text("ws only", status=400)
    while True:
        event = await receive()
        if event["type"] == "websocket.connect":
            await send({"type": "websocket.accept"})
        elif event["type"] == "websocket.receive":
            message = event["text"]

            async for chunk in await openai.ChatCompletion.acreate(
                model="gpt-3.5-turbo",
                messages=[{
                    "role": "user",
                    "content": message,
                }],
                stream=True,
            ):
                content = chunk["choices"][0].get("delta", {}).get("content")
                if content is not None:
                    await send({"type": "websocket.send", "text": content})

        elif event["type"] == "websocket.disconnect":
            break


def chat():
    return Response.html(CHAT)


@hookimpl
def register_routes():
    return [
        (r"^/ws$", websocket_application),
        (r"^/chat$", chat),
    ]

I put that in llm/plugin.py and then put this in setup.py:

    entry_points={
        "datasette": ["llm = llm.plugin"],
        "console_scripts": ["llm=llm.cli:cli"],
    },

And this in cli.py:

@cli.command()
def web():
    from datasette.app import Datasette
    import uvicorn

    path = get_log_db_path()
    if not os.path.exists(path):
        sqlite_utils.Database(path).vacuum()
    ds = Datasette(
        [path],
        metadata={
            "databases": {
                "log": {
                    "tables": {
                        "log": {
                            "sort_desc": "rowid",
                        }
                    }
                }
            }
        },
    )
    uvicorn.run(ds.app(), host="0.0.0.0", port=8302)

simonw · 2023-07-04T03:17:56Z

Mucked around with HTML and CSS a bit and got to this prototype:

<div class="chat-container">
  <div class="chat-bubble one">
    <div>
        <p>Hello, how are you?</p>
    </div>
    <img class="avatar" src="https://placekitten.com/40/40" alt="Person A Avatar">
  </div>
  <div class="chat-bubble two">
    <div>
        <p>I'm good, thanks! And you?</p>
    </div>
    <img class="avatar" src="https://placekitten.com/40/40" alt="Person B Avatar">
  </div>
  <div class="chat-bubble one">
    <div>
        <p>Lorem ipsum dolor sit amet consectetur, adipisicing elit. Distinctio similique quos ratione omnis impedit, est mollitia amet</p><p>aspernatur inventore consectetur, autem dolorum at nemo! Voluptas modi eveniet culpa nobis id?
    </div>
    <img class="avatar" src="https://placekitten.com/40/40" alt="Person A Avatar">
  </div>
  <div class="chat-bubble two">
    <div id="animatedText">
        <p>Lorem ipsum dolor sit amet consectetur, adipisicing elit. Distinctio similique quos ratione omnis impedit, est mollitia amet</p><p>aspernatur inventore consectetur, autem dolorum at nemo! Voluptas modi eveniet culpa nobis id?
    </div>
    <img class="avatar" src="https://placekitten.com/40/40" alt="Person B Avatar">
  </div>
</div>
<style>
.chat-container {
    display: flex;
    flex-direction: column;
    align-items: flex-start;
    font-family: Helvetica, sans-serif;
    line-height: 1.35;
    color: rgba(0, 0, 0, 0.8);
    max-width: 600px;

}

.chat-bubble {
    border-radius: 10px;
    padding: 10px;
    margin: 10px;
    width: 85%;
    border: 1px solid #ccc;
    background-color: #e6e5ff;

    display: flex;
    align-items: start;
}
.chat-bubble.one {
    border-color: #b9b7f2;
}
.chat-bubble.two {
    /* darker darker green */
    border-color: #98d798;
}

.chat-bubble.one img.avatar {
    order: -1;
    margin-right: 10px;
}

.chat-bubble.two {
    background-color: #ccffcc;
    align-self: flex-end;
    justify-content: space-between;
}

.chat-bubble.two img.avatar {
    order: 1;
    margin-left: 10px;
}

.chat-bubble p {
    margin-top: 0;
}
.chat-bubble p:last-of-type {
    margin-bottom: 0;
}

</style>

<script>
var text = "Lorem ipsum dolor sit amet consectetur, adipisicing elit. Distinctio similique quos ratione omnis impedit, est mollitia amet aspernatur inventore consectetur, autem dolorum at nemo! Voluptas modi eveniet culpa nobis id?";
var words = text.split(" ");
var container = document.getElementById("animatedText");
container.innerHTML = "";

function addWord(index) {
if (index < words.length) {
    container.innerHTML += words[index] + " ";
    setTimeout(function() {
    addWord(index + 1);
    }, 50);
}
}

addWord(0);
</script>

simonw · 2023-07-04T03:25:17Z

Added a submit form:

<div class="chat-container">
  <div class="chat-bubble one">
    <div>
        <p>Hello, how are you?</p>
    </div>
    <img class="avatar" src="https://placekitten.com/40/40" alt="Person A Avatar">
  </div>
  <div class="chat-bubble two">
    <div>
        <p>I'm good, thanks! And you?</p>
    </div>
    <img class="avatar" src="https://placekitten.com/40/40" alt="Person B Avatar">
  </div>
  <div class="chat-bubble one">
    <div>
        <p>Lorem ipsum dolor sit amet consectetur, adipisicing elit. Distinctio similique quos ratione omnis impedit, est mollitia amet</p><p>aspernatur inventore consectetur, autem dolorum at nemo! Voluptas modi eveniet culpa nobis id?
    </div>
    <img class="avatar" src="https://placekitten.com/40/40" alt="Person A Avatar">
  </div>
  <div class="chat-bubble two">
    <div id="animatedText">
        <p>Lorem ipsum dolor sit amet consectetur, adipisicing elit. Distinctio similique quos ratione omnis impedit, est mollitia amet</p><p>aspernatur inventore consectetur, autem dolorum at nemo! Voluptas modi eveniet culpa nobis id?
    </div>
    <img class="avatar" src="https://placekitten.com/40/40" alt="Person B Avatar">
  </div>
  <div class="chat-bubble one">
    <div class="contains-textarea">
        <form action="">
        <textarea>Type here</textarea>
        <p class="submit"><input type="submit" value="Send"></p>
        </form>
    </div>
    <img class="avatar" src="https://placekitten.com/40/40" alt="Person A Avatar">
  </div>

</div>
<style>
.chat-container form {
    margin: 0;
}
.contains-textarea {
    /* flex box should take all available width */
    flex: 1;
}
p.submit {
    text-align: right;
    padding-top: 5px;
}
p.submit input {
    border: 2px solid #7572db;
    padding: 3 10px;
    background-color: #b9b7f2;
}
textarea {
    width: 100%;
    padding: 5px;
    min-height: 60px;   
}
.chat-container {
    display: flex;
    flex-direction: column;
    align-items: flex-start;
    font-family: Helvetica, sans-serif;
    line-height: 1.35;
    color: rgba(0, 0, 0, 0.8);
    max-width: 600px;

}

.chat-bubble {
    border-radius: 10px;
    padding: 10px;
    margin: 10px;
    width: 85%;
    border: 1px solid #ccc;
    background-color: #e6e5ff;

    display: flex;
    align-items: start;
}
.chat-bubble.one {
    border-color: #b9b7f2;
}
.chat-bubble.two {
    /* darker darker green */
    border-color: #98d798;
}

.chat-bubble.one img.avatar {
    order: -1;
    margin-right: 10px;
}

.chat-bubble.two {
    background-color: #ccffcc;
    align-self: flex-end;
    justify-content: space-between;
}

.chat-bubble.two img.avatar {
    order: 1;
    margin-left: 10px;
}

.chat-bubble p {
    margin-top: 0;
}
.chat-bubble p:last-of-type {
    margin-bottom: 0;
}

</style>

<script>
var text = "Lorem ipsum dolor sit amet consectetur, adipisicing elit. Distinctio similique quos ratione omnis impedit, est mollitia amet aspernatur inventore consectetur, autem dolorum at nemo! Voluptas modi eveniet culpa nobis id?";
var words = text.split(" ");
var container = document.getElementById("animatedText");
container.innerHTML = "";

function addWord(index) {
if (index < words.length) {
    container.innerHTML += words[index] + " ";
    setTimeout(function() {
    addWord(index + 1);
    }, 50);
}
}

addWord(0);
</script>

simonw · 2023-07-10T18:29:12Z

I built a quick ASGI prototype demonstrating server-sent events here: https://gist.github.com/simonw/d3d4773666b863e628b1a60d5a20294d

simonw · 2023-07-14T20:45:19Z

Pushed a prototype to the web branch.

Currently needs a OPENAI_API_KEY environment variable. I still need to port it to using llm directly: https://llm.datasette.io/en/stable/python-api.html

JBX028 · 2023-07-17T08:37:04Z

Hi,
First of all, thanks a lot for having created LLM as well as the other CLI tools.
Would be so useful to have something like this in order to interact with LLM with http or websockets requests. I am a nodejs developer and could easily integrate any of the llm engines (since the last version) in my apps without to manage the complex installation of the various llm engines. Any ETA about this feature?

The code for this is pretty messy but it works. Refs #4

thiswillbeyourgithub · 2024-04-13T11:35:36Z

Why not just use gradio? In about 50 lines of code you could have tabs for a chat interface, displaying and interacting with your logs etc.

I've been using it for a while and is terrific

simonw added the enhancement New feature or request label Apr 2, 2023

simonw mentioned this issue Apr 2, 2023

Mechanism for continuing an existing conversation #6

Closed

simonw added a commit that referenced this issue Jul 14, 2023

Initial llm web prototype, refs #4

5bd5595

simonw added this to the 0.6 milestone Jul 15, 2023

simonw added a commit that referenced this issue Aug 2, 2023

Run prompts using selected model, executing in a thread, refs #4

b78cd9b

simonw added a commit that referenced this issue Aug 2, 2023

Initial llm web prototype, refs #4

5cb4c4a

simonw added a commit that referenced this issue Aug 2, 2023

Run prompts using selected model, executing in a thread, refs #4

14efa50

simonw added a commit that referenced this issue Aug 2, 2023

Clear button, refs #4

ea9a31d

simonw added a commit that referenced this issue Aug 2, 2023

Show detailed model name in options box, refs #4

ddcdd84

simonw added a commit that referenced this issue Aug 2, 2023

Log finished response to database

fda7978

The code for this is pretty messy but it works. Refs #4

simonw added a commit that referenced this issue Aug 2, 2023

Ability to set system prompt, refs #4

dd7036b

simonw added a commit that referenced this issue Aug 2, 2023

Initial --help for llm web, refs #4

93a5afd

simonw mentioned this issue Aug 12, 2023

Interface for running a prompt against a selected model simonw/datasette-llm#1

Open

simonw modified the milestones: 0.6, 0.8 Aug 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

llm web command - launches a web server #4

llm web command - launches a web server #4

simonw commented Apr 2, 2023

simonw commented Apr 2, 2023

simonw commented Apr 2, 2023

simonw commented Apr 2, 2023

simonw commented Jul 4, 2023

simonw commented Jul 4, 2023

simonw commented Jul 10, 2023

simonw commented Jul 14, 2023

JBX028 commented Jul 17, 2023

thiswillbeyourgithub commented Apr 13, 2024

llm web command - launches a web server #4

llm web command - launches a web server #4

Comments

simonw commented Apr 2, 2023

simonw commented Apr 2, 2023

simonw commented Apr 2, 2023

simonw commented Apr 2, 2023

simonw commented Jul 4, 2023

simonw commented Jul 4, 2023

simonw commented Jul 10, 2023

simonw commented Jul 14, 2023

JBX028 commented Jul 17, 2023

thiswillbeyourgithub commented Apr 13, 2024