tm_back/pkg/ollama/README.md

Ollama SDK

A comprehensive Go SDK for interacting with Ollama AI models. This SDK provides a clean, type-safe interface for text generation, chat conversations, embeddings, and model management.

Features

• Text Generation: Generate text using various AI models
• Chat Conversations: Interactive chat with support for system prompts and conversation history
• Embeddings: Generate embeddings for text similarity and search
• Model Management: Pull, list, delete, and inspect models
• Streaming Support: Real-time streaming of generated text and chat responses
• Automatic Retries: Configurable retry mechanism with exponential backoff
• Structured Logging: Built-in logging with structured fields
• Fluent API: Builder pattern for constructing complex chat conversations
• Type Safety: Full type safety with comprehensive validation
• Error Handling: Detailed error types and proper error wrapping
• Configuration: Flexible configuration via environment variables or code
• Health Checks: Built-in health checking capabilities

Installation

The SDK is already part of your tm module. Import it as:

import "tm/pkg/ollama"

Quick Start

Basic Setup

// 1. Configuration
config := ollama.DefaultConfig()
config.BaseURL = "http://localhost:11434" // Your Ollama server URL
config.DefaultModel = "llama2" // Default model to use

// 2. Logger (using project's logger)
loggerConfig := &logger.Config{Level: "info", Format: "json", Output: "stdout"}
projectLogger := logger.NewLogger(loggerConfig)

// 3. Create SDK
sdk, err := ollama.New(config, projectLogger)
if err != nil {
    log.Fatal("Failed to create Ollama SDK:", err)
}

Environment Variables

Set environment variables for automatic configuration:

export OLLAMA_BASE_URL="http://localhost:11434"
export OLLAMA_TIMEOUT="300s"
export OLLAMA_RETRY_ATTEMPTS="3"
export OLLAMA_RETRY_DELAY="2s"
export OLLAMA_ENABLE_LOGGING="true"
export OLLAMA_DEFAULT_MODEL="llama2"
export OLLAMA_DEFAULT_TEMPERATURE="0.7"
export OLLAMA_DEFAULT_TOP_P="0.9"
export OLLAMA_DEFAULT_MAX_TOKENS="1000"

API Reference

Configuration

type Config struct {
    BaseURL            string        `env:"OLLAMA_BASE_URL"`
    Timeout            time.Duration `env:"OLLAMA_TIMEOUT"`
    RetryAttempts      int           `env:"OLLAMA_RETRY_ATTEMPTS"`
    RetryDelay         time.Duration `env:"OLLAMA_RETRY_DELAY"`
    EnableLogging      bool          `env:"OLLAMA_ENABLE_LOGGING"`
    UserAgent          string        `env:"OLLAMA_USER_AGENT"`
    DefaultModel       string        `env:"OLLAMA_DEFAULT_MODEL"`
    DefaultTemperature float64       `env:"OLLAMA_DEFAULT_TEMPERATURE"`
    DefaultTopP        float64       `env:"OLLAMA_DEFAULT_TOP_P"`
    DefaultMaxTokens   int           `env:"OLLAMA_DEFAULT_MAX_TOKENS"`
    EnableStreaming    bool          `env:"OLLAMA_ENABLE_STREAMING"`
}

Creating SDK Instance

// With default configuration
sdk, err := ollama.New(nil, logger)

// With custom configuration
config := &ollama.Config{
    BaseURL:            "http://localhost:11434",
    Timeout:            300 * time.Second,
    RetryAttempts:      3,
    RetryDelay:         2 * time.Second,
    EnableLogging:      true,
    DefaultModel:       "llama2",
    DefaultTemperature: 0.7,
    DefaultTopP:        0.9,
    DefaultMaxTokens:   1000,
}
sdk, err := ollama.New(config, logger)

// With custom HTTP client
httpClient := &http.Client{Timeout: 10 * time.Second}
sdk, err := ollama.NewWithClient(config, httpClient, logger)

Usage Examples

Simple Text Generation

ctx := context.Background()

// Quick generation with default model
response, err := sdk.QuickGenerate(ctx, "Write a short story about a robot")
if err != nil {
    log.Printf("Generation failed: %v", err)
}
fmt.Println(response)

// Generation with specific model
response, err := sdk.QuickGenerateWithModel(ctx, "codellama", "Write a Python function to sort a list")

Advanced Text Generation

req := &ollama.GenerateRequest{
    Model:  "llama2",
    Prompt: "Explain quantum computing in simple terms",
    Options: map[string]interface{}{
        "temperature": 0.7,
        "top_p":      0.9,
        "max_tokens": 500,
    },
}

resp, err := sdk.Generate(ctx, req)
if err != nil {
    log.Printf("Generation failed: %v", err)
}
fmt.Printf("Generated: %s\n", resp.Response)

Chat Conversations

// Simple chat
response, err := sdk.QuickChat(ctx, "What is the capital of France?")
fmt.Println(response)

// Chat with system prompt
response, err := sdk.QuickChatWithSystem(ctx, 
    "You are a helpful coding assistant.", 
    "How do I implement a binary search in Go?")

Advanced Chat with Conversation History

req := &ollama.ChatRequest{
    Model: "llama2",
    Messages: []ollama.Message{
        {
            Role:    ollama.MessageRoleSystem,
            Content: "You are a helpful assistant.",
        },
        {
            Role:    ollama.MessageRoleUser,
            Content: "Hello, how are you?",
        },
        {
            Role:    ollama.MessageRoleAssistant,
            Content: "I'm doing well, thank you! How can I help you today?",
        },
        {
            Role:    ollama.MessageRoleUser,
            Content: "Can you help me with a coding problem?",
        },
    },
}

resp, err := sdk.Chat(ctx, req)
fmt.Printf("Assistant: %s\n", resp.Message.Content)

Fluent Chat Builder

response, err := sdk.NewChatBuilder().
    SetModel("llama2").
    SetSystem("You are a helpful coding assistant.").
    AddMessage(ollama.MessageRoleUser, "How do I create a struct in Go?").
    SetTemperature(0.7).
    SetMaxTokens(500).
    Send(ctx)

fmt.Printf("Assistant: %s\n", response.Message.Content)

// Continue the conversation
response, err = sdk.NewChatBuilder().
    SetModel("llama2").
    SetSystem("You are a helpful coding assistant.").
    AddMessage(ollama.MessageRoleUser, "How do I create a struct in Go?").
    SetTemperature(0.7).
    SetMaxTokens(500).
    Continue(ctx, "Can you show me an example with methods?")

fmt.Printf("Assistant: %s\n", response.Message.Content)

Streaming Responses

// Streaming text generation
err := sdk.StreamGenerate(ctx, &ollama.GenerateRequest{
    Model:  "llama2",
    Prompt: "Write a poem about programming",
}, func(chunk string) error {
    fmt.Print(chunk)
    return nil
})

// Streaming chat
err := sdk.StreamChat(ctx, &ollama.ChatRequest{
    Model: "llama2",
    Messages: []ollama.Message{
        {
            Role:    ollama.MessageRoleUser,
            Content: "Tell me a story",
        },
    },
}, func(chunk string) error {
    fmt.Print(chunk)
    return nil
})

Embeddings

// Generate embeddings
embeddings, err := sdk.QuickEmbed(ctx, "This is some text to embed")
if err != nil {
    log.Printf("Embedding failed: %v", err)
}

// Advanced embedding request
req := &ollama.EmbedRequest{
    Model:  "llama2",
    Prompt: "Text to generate embeddings for",
}

resp, err := sdk.Embed(ctx, req)
fmt.Printf("Embedding dimension: %d\n", len(resp.Embedding))

Model Management

// List available models
models, err := sdk.ListModels(ctx)
for _, model := range models.Models {
    fmt.Printf("Model: %s, Size: %d bytes\n", model.Name, model.Size)
}

// Pull a new model
pullResp, err := sdk.PullModel(ctx, "codellama")
fmt.Printf("Pull status: %s\n", pullResp.Status)

// Show model information
modelInfo, err := sdk.ShowModel(ctx, "llama2")
fmt.Printf("Model details: %+v\n", modelInfo.Details)

// Delete a model
deleteResp, err := sdk.DeleteModel(ctx, "unwanted-model")
fmt.Printf("Delete status: %s\n", deleteResp.Status)

Health Check

err := sdk.Health(ctx)
if err != nil {
    log.Printf("Ollama service is not available: %v", err)
} else {
    fmt.Println("Ollama service is healthy")
}

Error Handling

The SDK provides specific error types for different failure scenarios:

resp, err := sdk.Generate(ctx, req)
if err != nil {
    switch e := err.(type) {
    case *ollama.ConfigError:
        fmt.Printf("Configuration error: %s\n", e.Message)
    case *ollama.ModelError:
        fmt.Printf("Model error for %s: %s\n", e.Model, e.Message)
    case *ollama.GenerationError:
        fmt.Printf("Generation error: %s\n", e.Message)
    case *ollama.ConnectionError:
        fmt.Printf("Connection error: %s\n", e.Message)
    case *ollama.TimeoutError:
        fmt.Printf("Timeout during %s: %s\n", e.Operation, e.Message)
    default:
        fmt.Printf("Unexpected error: %v\n", err)
    }
}

Advanced Usage

Custom HTTP Client

httpClient := &http.Client{
    Timeout: 60 * time.Second,
    Transport: &http.Transport{
        MaxIdleConns:        10,
        IdleConnTimeout:     30 * time.Second,
        DisableCompression:  true,
    },
}

sdk, err := ollama.NewWithClient(config, httpClient, logger)

Custom Options

req := &ollama.GenerateRequest{
    Model:  "llama2",
    Prompt: "Write a haiku",
}

opts := &ollama.GenerateOptions{
    Temperature:   &[]float64{0.5}[0],
    TopP:         &[]float64{0.8}[0],
    TopK:         &[]int{40}[0],
    RepeatPenalty: &[]float64{1.1}[0],
    Seed:         &[]int{42}[0],
    Stop:         []string{"</s>", "\n\n"},
    Format:       "json",
}

resp, err := sdk.GenerateWithOptions(ctx, req, opts)

Best Practices

1. Always use context: Pass context for cancellation and timeouts
2. Handle errors properly: Use type assertions for specific error handling
3. Configure timeouts: Set appropriate timeouts for your use case
4. Use streaming for long responses: Enable streaming for better user experience
5. Monitor model usage: Keep track of token usage and costs
6. Validate inputs: Ensure prompts and parameters are within limits
7. Use structured logging: Leverage the built-in logging for debugging

Troubleshooting

Common Issues

1. Connection refused: Ensure Ollama server is running on the correct port
2. Model not found: Pull the model first using PullModel()
3. Timeout errors: Increase the timeout configuration
4. Rate limiting: Implement backoff strategies for high-volume usage

Debug Mode

Enable debug logging to troubleshoot issues:

config := ollama.DefaultConfig()
config.EnableLogging = true

loggerConfig := &logger.Config{
    Level:  "debug",
    Format: "json",
    Output: "stdout",
}
projectLogger := logger.NewLogger(loggerConfig)

sdk, err := ollama.New(config, projectLogger)

License

This SDK is part of the Tender Management system and follows the same licensing terms.