tm_back/pkg/config/README.md

Modular Configuration System

This package provides a modular configuration system for the Tender Management backend. It allows each command (web, scraper, worker, etc.) to have its own specific configuration while sharing common base configurations.

Architecture

Base Configuration (BaseConfig)

The BaseConfig struct contains common configuration fields that are used across all commands:

• Server: HTTP server configuration (host, port, timeouts)
• Database: MongoDB connection configuration
• Cache: Redis connection configuration
• Logging: Logging system configuration

Command-Specific Configuration

Each command can extend the BaseConfig by embedding it and adding its own specific configuration fields.

Usage

1. Create Command-Specific Config

Create a config file for your command (e.g., cmd/yourcommand/config.go):

package main

import (
    "time"
    "tm/pkg/config"
)

// Config holds configuration for your command
type Config struct {
    config.BaseConfig
    YourSpecific YourSpecificConfig `mapstructure:"your_specific"`
}

type YourSpecificConfig struct {
    SomeField    string        `mapstructure:"some_field"`
    SomeTimeout  time.Duration `mapstructure:"some_timeout"`
    SomeNumber   int           `mapstructure:"some_number"`
}

2. Load Configuration

In your command's main.go or bootstrap file:

func initConfig() Config {
    conf, err := config.LoadConfig(".", &Config{})
    if err != nil {
        panic(fmt.Sprintf("Failed to load config: %v", err))
    }
    
    return *conf
}

3. Create YAML Configuration File

Create a config.yaml file in your command directory:

# Base configuration (common to all commands)
server:
  host: "localhost"
  port: 8080
  timeout: 30s
  read_timeout: 10s
  write_timeout: 10s

database:
  mongodb:
    uri: "mongodb://localhost:27017"
    name: "your_database"
    timeout: 10s
    max_pool_size: 100

cache:
  redis:
    host: "localhost"
    port: 6379
    password: ""
    db: 0
    pool_size: 10

logging:
  level: "info"
  format: "json"
  output: "stdout"
  file:
    path: "/var/log/tm/your-command.log"
    max_size: 100
    max_backups: 5
    max_age: 30
    compress: true

# Your command-specific configuration
your_specific:
  some_field: "value"
  some_timeout: 30s
  some_number: 42

Examples

Web Command Config

type Config struct {
    config.BaseConfig
    Queue                 QueueConfig     `mapstructure:"queue"`
    UserAuthorization     AuthConfig      `mapstructure:"user_authorization"`
    CustomerAuthorization AuthConfig      `mapstructure:"customer_authorization"`
    AI                    AIConfig        `mapstructure:"ai"`
    RateLimit             RateLimitConfig `mapstructure:"rate_limiting"`
}

Scraper Command Config

type Config struct {
    config.BaseConfig
    TED TEDConfig `mapstructure:"ted"`
}

type TEDConfig struct {
    BaseURL          string        `mapstructure:"base_url"`
    MaxRetries       int           `mapstructure:"max_retries"`
    MaxConcurrency   int           `mapstructure:"max_concurrency"`
    DownloadDir      string        `mapstructure:"download_dir"`
    CleanupAfter     time.Duration `mapstructure:"cleanup_after"`
}

Worker Command Config

type Config struct {
    config.BaseConfig
    Queue    QueueConfig    `mapstructure:"queue"`
    AI       AIConfig       `mapstructure:"ai"`
    Worker   WorkerConfig   `mapstructure:"worker"`
    Scraping ScrapingConfig `mapstructure:"scraping"`
}

Features

Type Safety

The configuration loader uses Go generics to provide type-safe configuration loading. The function signature ensures that you get back exactly the type you expect.

Environment Variable Support

The configuration system automatically supports environment variable overrides via Viper's AutomaticEnv() feature.

YAML Support

Configuration files use YAML format for easy readability and maintenance.

Validation

Each command can implement its own validation logic after loading the configuration.

Best Practices

1. Keep BaseConfig minimal: Only add fields to BaseConfig that are truly common across all commands.

2. Use descriptive names: Choose clear, descriptive names for your configuration structs and fields.

3. Document your config: Add comments to your configuration structs explaining what each field does.

4. Use appropriate types: Use time.Duration for durations, proper numeric types for numbers, etc.

5. Provide defaults: Consider providing sensible defaults in your YAML files.

6. Environment variables: Use environment variables for sensitive data like API keys and passwords.

Migration from Old System

If you're migrating from the old infra.Config system:

1. Remove imports of tm/infra
2. Add imports of tm/pkg/config
3. Change your config struct to embed config.BaseConfig
4. Update your LoadConfig call to use the new generic function
5. Update function signatures to use the new config types

Example migration:

// Old way
func initConfig() infra.Config {
    config, err := infra.LoadConfig(".")
    if err != nil {
        panic(fmt.Sprintf("Failed to load config: %v", err))
    }
    return *config
}

// New way
func initConfig() Config {
    conf, err := config.LoadConfig(".", &Config{})
    if err != nil {
        panic(fmt.Sprintf("Failed to load config: %v", err))
    }
    return *conf
}