# Ollama SDK A comprehensive Go SDK for interacting with Ollama AI models. This SDK provides a clean, type-safe interface for text generation, chat conversations, embeddings, and model management. ## Features - **Text Generation**: Generate text using various AI models - **Chat Conversations**: Interactive chat with support for system prompts and conversation history - **Embeddings**: Generate embeddings for text similarity and search - **Model Management**: Pull, list, delete, and inspect models - **Streaming Support**: Real-time streaming of generated text and chat responses - **Automatic Retries**: Configurable retry mechanism with exponential backoff - **Structured Logging**: Built-in logging with structured fields - **Fluent API**: Builder pattern for constructing complex chat conversations - **Type Safety**: Full type safety with comprehensive validation - **Error Handling**: Detailed error types and proper error wrapping - **Configuration**: Flexible configuration via environment variables or code - **Health Checks**: Built-in health checking capabilities ## Installation The SDK is already part of your `tm` module. Import it as: ```go import "tm/pkg/ollama" ``` ## Quick Start ### Basic Setup ```go // 1. Configuration config := ollama.DefaultConfig() config.BaseURL = "http://localhost:11434" // Your Ollama server URL config.DefaultModel = "llama2" // Default model to use // 2. Logger (using project's logger) loggerConfig := &logger.Config{Level: "info", Format: "json", Output: "stdout"} projectLogger := logger.NewLogger(loggerConfig) // 3. Create SDK sdk, err := ollama.New(config, projectLogger) if err != nil { log.Fatal("Failed to create Ollama SDK:", err) } ``` ### Environment Variables Set environment variables for automatic configuration: ```bash export OLLAMA_BASE_URL="http://localhost:11434" export OLLAMA_TIMEOUT="300s" export OLLAMA_RETRY_ATTEMPTS="3" export OLLAMA_RETRY_DELAY="2s" export OLLAMA_ENABLE_LOGGING="true" export OLLAMA_DEFAULT_MODEL="llama2" export OLLAMA_DEFAULT_TEMPERATURE="0.7" export OLLAMA_DEFAULT_TOP_P="0.9" export OLLAMA_DEFAULT_MAX_TOKENS="1000" ``` ## API Reference ### Configuration ```go type Config struct { BaseURL string `env:"OLLAMA_BASE_URL"` Timeout time.Duration `env:"OLLAMA_TIMEOUT"` RetryAttempts int `env:"OLLAMA_RETRY_ATTEMPTS"` RetryDelay time.Duration `env:"OLLAMA_RETRY_DELAY"` EnableLogging bool `env:"OLLAMA_ENABLE_LOGGING"` UserAgent string `env:"OLLAMA_USER_AGENT"` DefaultModel string `env:"OLLAMA_DEFAULT_MODEL"` DefaultTemperature float64 `env:"OLLAMA_DEFAULT_TEMPERATURE"` DefaultTopP float64 `env:"OLLAMA_DEFAULT_TOP_P"` DefaultMaxTokens int `env:"OLLAMA_DEFAULT_MAX_TOKENS"` EnableStreaming bool `env:"OLLAMA_ENABLE_STREAMING"` } ``` ### Creating SDK Instance ```go // With default configuration sdk, err := ollama.New(nil, logger) // With custom configuration config := &ollama.Config{ BaseURL: "http://localhost:11434", Timeout: 300 * time.Second, RetryAttempts: 3, RetryDelay: 2 * time.Second, EnableLogging: true, DefaultModel: "llama2", DefaultTemperature: 0.7, DefaultTopP: 0.9, DefaultMaxTokens: 1000, } sdk, err := ollama.New(config, logger) // With custom HTTP client httpClient := &http.Client{Timeout: 10 * time.Second} sdk, err := ollama.NewWithClient(config, httpClient, logger) ``` ## Usage Examples ### Simple Text Generation ```go ctx := context.Background() // Quick generation with default model response, err := sdk.QuickGenerate(ctx, "Write a short story about a robot") if err != nil { log.Printf("Generation failed: %v", err) } fmt.Println(response) // Generation with specific model response, err := sdk.QuickGenerateWithModel(ctx, "codellama", "Write a Python function to sort a list") ``` ### Advanced Text Generation ```go req := &ollama.GenerateRequest{ Model: "llama2", Prompt: "Explain quantum computing in simple terms", Options: map[string]interface{}{ "temperature": 0.7, "top_p": 0.9, "max_tokens": 500, }, } resp, err := sdk.Generate(ctx, req) if err != nil { log.Printf("Generation failed: %v", err) } fmt.Printf("Generated: %s\n", resp.Response) ``` ### Chat Conversations ```go // Simple chat response, err := sdk.QuickChat(ctx, "What is the capital of France?") fmt.Println(response) // Chat with system prompt response, err := sdk.QuickChatWithSystem(ctx, "You are a helpful coding assistant.", "How do I implement a binary search in Go?") ``` ### Advanced Chat with Conversation History ```go req := &ollama.ChatRequest{ Model: "llama2", Messages: []ollama.Message{ { Role: ollama.MessageRoleSystem, Content: "You are a helpful assistant.", }, { Role: ollama.MessageRoleUser, Content: "Hello, how are you?", }, { Role: ollama.MessageRoleAssistant, Content: "I'm doing well, thank you! How can I help you today?", }, { Role: ollama.MessageRoleUser, Content: "Can you help me with a coding problem?", }, }, } resp, err := sdk.Chat(ctx, req) fmt.Printf("Assistant: %s\n", resp.Message.Content) ``` ### Fluent Chat Builder ```go response, err := sdk.NewChatBuilder(). SetModel("llama2"). SetSystem("You are a helpful coding assistant."). AddMessage(ollama.MessageRoleUser, "How do I create a struct in Go?"). SetTemperature(0.7). SetMaxTokens(500). Send(ctx) fmt.Printf("Assistant: %s\n", response.Message.Content) // Continue the conversation response, err = sdk.NewChatBuilder(). SetModel("llama2"). SetSystem("You are a helpful coding assistant."). AddMessage(ollama.MessageRoleUser, "How do I create a struct in Go?"). SetTemperature(0.7). SetMaxTokens(500). Continue(ctx, "Can you show me an example with methods?") fmt.Printf("Assistant: %s\n", response.Message.Content) ``` ### Streaming Responses ```go // Streaming text generation err := sdk.StreamGenerate(ctx, &ollama.GenerateRequest{ Model: "llama2", Prompt: "Write a poem about programming", }, func(chunk string) error { fmt.Print(chunk) return nil }) // Streaming chat err := sdk.StreamChat(ctx, &ollama.ChatRequest{ Model: "llama2", Messages: []ollama.Message{ { Role: ollama.MessageRoleUser, Content: "Tell me a story", }, }, }, func(chunk string) error { fmt.Print(chunk) return nil }) ``` ### Embeddings ```go // Generate embeddings embeddings, err := sdk.QuickEmbed(ctx, "This is some text to embed") if err != nil { log.Printf("Embedding failed: %v", err) } // Advanced embedding request req := &ollama.EmbedRequest{ Model: "llama2", Prompt: "Text to generate embeddings for", } resp, err := sdk.Embed(ctx, req) fmt.Printf("Embedding dimension: %d\n", len(resp.Embedding)) ``` ### Model Management ```go // List available models models, err := sdk.ListModels(ctx) for _, model := range models.Models { fmt.Printf("Model: %s, Size: %d bytes\n", model.Name, model.Size) } // Pull a new model pullResp, err := sdk.PullModel(ctx, "codellama") fmt.Printf("Pull status: %s\n", pullResp.Status) // Show model information modelInfo, err := sdk.ShowModel(ctx, "llama2") fmt.Printf("Model details: %+v\n", modelInfo.Details) // Delete a model deleteResp, err := sdk.DeleteModel(ctx, "unwanted-model") fmt.Printf("Delete status: %s\n", deleteResp.Status) ``` ### Health Check ```go err := sdk.Health(ctx) if err != nil { log.Printf("Ollama service is not available: %v", err) } else { fmt.Println("Ollama service is healthy") } ``` ## Error Handling The SDK provides specific error types for different failure scenarios: ```go resp, err := sdk.Generate(ctx, req) if err != nil { switch e := err.(type) { case *ollama.ConfigError: fmt.Printf("Configuration error: %s\n", e.Message) case *ollama.ModelError: fmt.Printf("Model error for %s: %s\n", e.Model, e.Message) case *ollama.GenerationError: fmt.Printf("Generation error: %s\n", e.Message) case *ollama.ConnectionError: fmt.Printf("Connection error: %s\n", e.Message) case *ollama.TimeoutError: fmt.Printf("Timeout during %s: %s\n", e.Operation, e.Message) default: fmt.Printf("Unexpected error: %v\n", err) } } ``` ## Advanced Usage ### Custom HTTP Client ```go httpClient := &http.Client{ Timeout: 60 * time.Second, Transport: &http.Transport{ MaxIdleConns: 10, IdleConnTimeout: 30 * time.Second, DisableCompression: true, }, } sdk, err := ollama.NewWithClient(config, httpClient, logger) ``` ### Custom Options ```go req := &ollama.GenerateRequest{ Model: "llama2", Prompt: "Write a haiku", } opts := &ollama.GenerateOptions{ Temperature: &[]float64{0.5}[0], TopP: &[]float64{0.8}[0], TopK: &[]int{40}[0], RepeatPenalty: &[]float64{1.1}[0], Seed: &[]int{42}[0], Stop: []string{"", "\n\n"}, Format: "json", } resp, err := sdk.GenerateWithOptions(ctx, req, opts) ``` ## Best Practices 1. **Always use context**: Pass context for cancellation and timeouts 2. **Handle errors properly**: Use type assertions for specific error handling 3. **Configure timeouts**: Set appropriate timeouts for your use case 4. **Use streaming for long responses**: Enable streaming for better user experience 5. **Monitor model usage**: Keep track of token usage and costs 6. **Validate inputs**: Ensure prompts and parameters are within limits 7. **Use structured logging**: Leverage the built-in logging for debugging ## Troubleshooting ### Common Issues 1. **Connection refused**: Ensure Ollama server is running on the correct port 2. **Model not found**: Pull the model first using `PullModel()` 3. **Timeout errors**: Increase the timeout configuration 4. **Rate limiting**: Implement backoff strategies for high-volume usage ### Debug Mode Enable debug logging to troubleshoot issues: ```go config := ollama.DefaultConfig() config.EnableLogging = true loggerConfig := &logger.Config{ Level: "debug", Format: "json", Output: "stdout", } projectLogger := logger.NewLogger(loggerConfig) sdk, err := ollama.New(config, projectLogger) ``` ## License This SDK is part of the Tender Management system and follows the same licensing terms.