mazyar/tm_back
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add Ollama SDK with Core Functionality and Documentation

Browse Source 
- Introduced the Ollama SDK, providing a comprehensive interface for interacting with Ollama AI models, including text generation, chat conversations, embeddings, and model management.
- Implemented core components such as Client, Config, and various request/response entities to facilitate API interactions.
- Added structured error handling with specific error types for improved clarity and debugging.
- Included a README and usage guide to assist developers in integrating the SDK into their applications, ensuring a smooth onboarding experience.
- Established a fluent API for chat interactions and streaming responses, enhancing usability and flexibility for developers.
This commit is contained in:
n.nakhostin
2025-10-04 15:21:35 +03:30
parent 0dd877a46d
commit 154610e2a0
8 changed files with 1719 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
pkg/ollama/README.md
+394
View File
@@ -0,0 +1,394 @@
# Ollama SDK
A comprehensive Go SDK for interacting with Ollama AI models. This SDK provides a clean, type-safe interface for text generation, chat conversations, embeddings, and model management.
## Features
- **Text Generation**: Generate text using various AI models
- **Chat Conversations**: Interactive chat with support for system prompts and conversation history
- **Embeddings**: Generate embeddings for text similarity and search
- **Model Management**: Pull, list, delete, and inspect models
- **Streaming Support**: Real-time streaming of generated text and chat responses
- **Automatic Retries**: Configurable retry mechanism with exponential backoff
- **Structured Logging**: Built-in logging with structured fields
- **Fluent API**: Builder pattern for constructing complex chat conversations
- **Type Safety**: Full type safety with comprehensive validation
- **Error Handling**: Detailed error types and proper error wrapping
- **Configuration**: Flexible configuration via environment variables or code
- **Health Checks**: Built-in health checking capabilities
## Installation
The SDK is already part of your `tm` module. Import it as:
```go
import "tm/pkg/ollama"
```
## Quick Start
### Basic Setup
```go
// 1. Configuration
config := ollama.DefaultConfig()
config.BaseURL = "http://localhost:11434" // Your Ollama server URL
config.DefaultModel = "llama2" // Default model to use
// 2. Logger (using project's logger)
loggerConfig := &logger.Config{Level: "info", Format: "json", Output: "stdout"}
projectLogger := logger.NewLogger(loggerConfig)
// 3. Create SDK
sdk, err := ollama.New(config, projectLogger)
if err != nil {
log.Fatal("Failed to create Ollama SDK:", err)
}
```
### Environment Variables
Set environment variables for automatic configuration:
```bash
export OLLAMA_BASE_URL="http://localhost:11434"
export OLLAMA_TIMEOUT="300s"
export OLLAMA_RETRY_ATTEMPTS="3"
export OLLAMA_RETRY_DELAY="2s"
export OLLAMA_ENABLE_LOGGING="true"
export OLLAMA_DEFAULT_MODEL="llama2"
export OLLAMA_DEFAULT_TEMPERATURE="0.7"
export OLLAMA_DEFAULT_TOP_P="0.9"
export OLLAMA_DEFAULT_MAX_TOKENS="1000"
```
## API Reference
### Configuration
```go
type Config struct {
BaseURL string `env:"OLLAMA_BASE_URL"`
Timeout time.Duration `env:"OLLAMA_TIMEOUT"`
RetryAttempts int `env:"OLLAMA_RETRY_ATTEMPTS"`
RetryDelay time.Duration `env:"OLLAMA_RETRY_DELAY"`
EnableLogging bool `env:"OLLAMA_ENABLE_LOGGING"`
UserAgent string `env:"OLLAMA_USER_AGENT"`
DefaultModel string `env:"OLLAMA_DEFAULT_MODEL"`
DefaultTemperature float64 `env:"OLLAMA_DEFAULT_TEMPERATURE"`
DefaultTopP float64 `env:"OLLAMA_DEFAULT_TOP_P"`
DefaultMaxTokens int `env:"OLLAMA_DEFAULT_MAX_TOKENS"`
EnableStreaming bool `env:"OLLAMA_ENABLE_STREAMING"`
}
```
### Creating SDK Instance
```go
// With default configuration
sdk, err := ollama.New(nil, logger)
// With custom configuration
config := &ollama.Config{
BaseURL: "http://localhost:11434",
Timeout: 300 * time.Second,
RetryAttempts: 3,
RetryDelay: 2 * time.Second,
EnableLogging: true,
DefaultModel: "llama2",
DefaultTemperature: 0.7,
DefaultTopP: 0.9,
DefaultMaxTokens: 1000,
}
sdk, err := ollama.New(config, logger)
// With custom HTTP client
httpClient := &http.Client{Timeout: 10 * time.Second}
sdk, err := ollama.NewWithClient(config, httpClient, logger)
```
## Usage Examples
### Simple Text Generation
```go
ctx := context.Background()
// Quick generation with default model
response, err := sdk.QuickGenerate(ctx, "Write a short story about a robot")
if err != nil {
log.Printf("Generation failed: %v", err)
}
fmt.Println(response)
// Generation with specific model
response, err := sdk.QuickGenerateWithModel(ctx, "codellama", "Write a Python function to sort a list")
```
### Advanced Text Generation
```go
req := &ollama.GenerateRequest{
Model: "llama2",
Prompt: "Explain quantum computing in simple terms",
Options: map[string]interface{}{
"temperature": 0.7,
"top_p": 0.9,
"max_tokens": 500,
},
}
resp, err := sdk.Generate(ctx, req)
if err != nil {
log.Printf("Generation failed: %v", err)
}
fmt.Printf("Generated: %s\n", resp.Response)
```
### Chat Conversations
```go
// Simple chat
response, err := sdk.QuickChat(ctx, "What is the capital of France?")
fmt.Println(response)
// Chat with system prompt
response, err := sdk.QuickChatWithSystem(ctx,
"You are a helpful coding assistant.",
"How do I implement a binary search in Go?")
```
### Advanced Chat with Conversation History
```go
req := &ollama.ChatRequest{
Model: "llama2",
Messages: []ollama.Message{
{
Role: ollama.MessageRoleSystem,
Content: "You are a helpful assistant.",
},
{
Role: ollama.MessageRoleUser,
Content: "Hello, how are you?",
},
{
Role: ollama.MessageRoleAssistant,
Content: "I'm doing well, thank you! How can I help you today?",
},
{
Role: ollama.MessageRoleUser,
Content: "Can you help me with a coding problem?",
},
},
}
resp, err := sdk.Chat(ctx, req)
fmt.Printf("Assistant: %s\n", resp.Message.Content)
```
### Fluent Chat Builder
```go
response, err := sdk.NewChatBuilder().
SetModel("llama2").
SetSystem("You are a helpful coding assistant.").
AddMessage(ollama.MessageRoleUser, "How do I create a struct in Go?").
SetTemperature(0.7).
SetMaxTokens(500).
Send(ctx)
fmt.Printf("Assistant: %s\n", response.Message.Content)
// Continue the conversation
response, err = sdk.NewChatBuilder().
SetModel("llama2").
SetSystem("You are a helpful coding assistant.").
AddMessage(ollama.MessageRoleUser, "How do I create a struct in Go?").
SetTemperature(0.7).
SetMaxTokens(500).
Continue(ctx, "Can you show me an example with methods?")
fmt.Printf("Assistant: %s\n", response.Message.Content)
```
### Streaming Responses
```go
// Streaming text generation
err := sdk.StreamGenerate(ctx, &ollama.GenerateRequest{
Model: "llama2",
Prompt: "Write a poem about programming",
}, func(chunk string) error {
fmt.Print(chunk)
return nil
})
// Streaming chat
err := sdk.StreamChat(ctx, &ollama.ChatRequest{
Model: "llama2",
Messages: []ollama.Message{
{
Role: ollama.MessageRoleUser,
Content: "Tell me a story",
},
},
}, func(chunk string) error {
fmt.Print(chunk)
return nil
})
```
### Embeddings
```go
// Generate embeddings
embeddings, err := sdk.QuickEmbed(ctx, "This is some text to embed")
if err != nil {
log.Printf("Embedding failed: %v", err)
}
// Advanced embedding request
req := &ollama.EmbedRequest{
Model: "llama2",
Prompt: "Text to generate embeddings for",
}
resp, err := sdk.Embed(ctx, req)
fmt.Printf("Embedding dimension: %d\n", len(resp.Embedding))
```
### Model Management
```go
// List available models
models, err := sdk.ListModels(ctx)
for _, model := range models.Models {
fmt.Printf("Model: %s, Size: %d bytes\n", model.Name, model.Size)
}
// Pull a new model
pullResp, err := sdk.PullModel(ctx, "codellama")
fmt.Printf("Pull status: %s\n", pullResp.Status)
// Show model information
modelInfo, err := sdk.ShowModel(ctx, "llama2")
fmt.Printf("Model details: %+v\n", modelInfo.Details)
// Delete a model
deleteResp, err := sdk.DeleteModel(ctx, "unwanted-model")
fmt.Printf("Delete status: %s\n", deleteResp.Status)
```
### Health Check
```go
err := sdk.Health(ctx)
if err != nil {
log.Printf("Ollama service is not available: %v", err)
} else {
fmt.Println("Ollama service is healthy")
}
```
## Error Handling
The SDK provides specific error types for different failure scenarios:
```go
resp, err := sdk.Generate(ctx, req)
if err != nil {
switch e := err.(type) {
case *ollama.ConfigError:
fmt.Printf("Configuration error: %s\n", e.Message)
case *ollama.ModelError:
fmt.Printf("Model error for %s: %s\n", e.Model, e.Message)
case *ollama.GenerationError:
fmt.Printf("Generation error: %s\n", e.Message)
case *ollama.ConnectionError:
fmt.Printf("Connection error: %s\n", e.Message)
case *ollama.TimeoutError:
fmt.Printf("Timeout during %s: %s\n", e.Operation, e.Message)
default:
fmt.Printf("Unexpected error: %v\n", err)
}
}
```
## Advanced Usage
### Custom HTTP Client
```go
httpClient := &http.Client{
Timeout: 60 * time.Second,
Transport: &http.Transport{
MaxIdleConns: 10,
IdleConnTimeout: 30 * time.Second,
DisableCompression: true,
},
}
sdk, err := ollama.NewWithClient(config, httpClient, logger)
```
### Custom Options
```go
req := &ollama.GenerateRequest{
Model: "llama2",
Prompt: "Write a haiku",
}
opts := &ollama.GenerateOptions{
Temperature: &[]float64{0.5}[0],
TopP: &[]float64{0.8}[0],
TopK: &[]int{40}[0],
RepeatPenalty: &[]float64{1.1}[0],
Seed: &[]int{42}[0],
Stop: []string{"</s>", "\n\n"},
Format: "json",
}
resp, err := sdk.GenerateWithOptions(ctx, req, opts)
```
## Best Practices
1. **Always use context**: Pass context for cancellation and timeouts
2. **Handle errors properly**: Use type assertions for specific error handling
3. **Configure timeouts**: Set appropriate timeouts for your use case
4. **Use streaming for long responses**: Enable streaming for better user experience
5. **Monitor model usage**: Keep track of token usage and costs
6. **Validate inputs**: Ensure prompts and parameters are within limits
7. **Use structured logging**: Leverage the built-in logging for debugging
## Troubleshooting
### Common Issues
1. **Connection refused**: Ensure Ollama server is running on the correct port
2. **Model not found**: Pull the model first using `PullModel()`
3. **Timeout errors**: Increase the timeout configuration
4. **Rate limiting**: Implement backoff strategies for high-volume usage
### Debug Mode
Enable debug logging to troubleshoot issues:
```go
config := ollama.DefaultConfig()
config.EnableLogging = true
loggerConfig := &logger.Config{
Level: "debug",
Format: "json",
Output: "stdout",
}
projectLogger := logger.NewLogger(loggerConfig)
sdk, err := ollama.New(config, projectLogger)
```
## License
This SDK is part of the Tender Management system and follows the same licensing terms.