tm_back/pkg/config/README.md

# Environment-Based Configuration System

This package provides a configuration system for the Tender Management backend that loads configuration from both OS environment variables and `.env` files, with **OS environment variables taking priority** over `.env` file values. It maintains the same structure as the previous YAML-based system while using environment variables for all configuration values.

## Priority System

The configuration loader follows this priority order:
1. **OS Environment Variables** (highest priority) - System-level environment variables
2. **`.env` File Values** (lower priority) - File-based configuration for development

This means that if you set `SERVER_PORT=8080` in your `.env` file but have `SERVER_PORT=9000` as an OS environment variable, the system will use `9000`.

## Architecture

### Base Configuration (`BaseConfig`)

The configuration system provides common configuration structs that are used across all commands:

- **ServerConfig**: HTTP server configuration (host, port, timeouts)
- **DatabaseConfig**: MongoDB connection configuration  
- **CacheConfig**: Redis connection configuration
- **LoggingConfig**: Logging system configuration
- **RateLimitConfig**: Rate limiting configuration

### Command-Specific Configuration

Each command can extend the base configuration by embedding the common configs and adding their own specific configuration fields.

## Usage

### 1. Create Command-Specific Config

Create a config file for your command (e.g., `cmd/yourcommand/config.go`):

```go
package main

import (
    "time"
    "tm/pkg/config"
)

// Config holds configuration for your command
type Config struct {
    config.ServerConfig
    config.DatabaseConfig
    config.CacheConfig
    config.LoggingConfig
    YourSpecific YourSpecificConfig
}

type YourSpecificConfig struct {
    SomeField    string        `env:"SOME_FIELD"`
    SomeTimeout  time.Duration `env:"SOME_TIMEOUT"`
    SomeNumber   int           `env:"SOME_NUMBER"`
}
```

### 2. Load Configuration

In your command's `main.go` or bootstrap file:

```go
func initConfig() Config {
    conf, err := config.LoadConfig(".", &Config{})
    if err != nil {
        panic(fmt.Sprintf("Failed to load config: %v", err))
    }
    
    return *conf
}
```

### 3. Configuration Sources

#### Option A: Using .env File Only (Development)

Create a `.env` file in your command directory:

```env
# Server Configuration
SERVER_HOST=localhost
SERVER_PORT=8080
SERVER_TIMEOUT=30s
SERVER_READ_TIMEOUT=10s
SERVER_WRITE_TIMEOUT=10s

# Database Configuration
MONGODB_URI=mongodb://localhost:27017
MONGODB_NAME=your_database
MONGODB_TIMEOUT=10s
MONGODB_MAX_POOL_SIZE=100

# Cache Configuration
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=
REDIS_DB=0
REDIS_POOL_SIZE=10

# Logging Configuration
LOG_LEVEL=info
LOG_FORMAT=json
LOG_OUTPUT=stdout
LOG_FILE_PATH=/var/log/tm/your-command.log
LOG_FILE_MAX_SIZE=100
LOG_FILE_MAX_BACKUPS=5
LOG_FILE_MAX_AGE=30
LOG_FILE_COMPRESS=true

# Your command-specific configuration
SOME_FIELD=value
SOME_TIMEOUT=30s
SOME_NUMBER=42
```

#### Option B: Using OS Environment Variables (Production)

```bash
# Set environment variables directly
export SERVER_HOST=0.0.0.0
export SERVER_PORT=8080
export MONGODB_URI=mongodb://mongo:27017
export MONGODB_NAME=production_db
export LOG_LEVEL=error

# Run your application
./your-command
```

#### Option C: Mixed Approach (Recommended)

1. Create a `.env` file with development defaults
2. Override specific values with OS environment variables for production:

```bash
# .env file contains development defaults
# Override critical values with OS env vars
export MONGODB_URI=mongodb://production-server:27017
export MONGODB_NAME=production_db
export LOG_LEVEL=error

# Run application - will use .env defaults except for overridden values
./your-command
```

## Priority Example

Given this `.env` file:
```env
SERVER_HOST=localhost
SERVER_PORT=8080
LOG_LEVEL=debug
```

And these OS environment variables:
```bash
export SERVER_PORT=9000
export LOG_LEVEL=error
```

The final configuration will be:
- `SERVER_HOST=localhost` (from .env file, no OS override)
- `SERVER_PORT=9000` (OS environment variable overrides .env)
- `LOG_LEVEL=error` (OS environment variable overrides .env)

## Advanced Features

### Checking Configuration Source

You can check whether a configuration value came from OS environment or .env file:

```go
import "tm/pkg/config"

func main() {
    value, isFromOS := config.GetEnvWithPriority("SERVER_PORT")
    if isFromOS {
        fmt.Printf("SERVER_PORT=%s (from OS environment)\n", value)
    } else {
        fmt.Printf("SERVER_PORT=%s (from .env file)\n", value)
    }
}
```

### Graceful .env File Handling

The system gracefully handles missing `.env` files:
- If `.env` file exists: loads values as defaults
- If `.env` file is missing: continues with OS environment variables only
- No errors are thrown for missing `.env` files

## Examples

### Web Command Config

```go
type Config struct {
    config.ServerConfig
    config.DatabaseConfig
    config.CacheConfig
    config.LoggingConfig
    config.RateLimitConfig
    UserAuth     AuthConfig
    CustomerAuth AuthConfig
    Assets       AssetsConfig
}

type AuthConfig struct {
    JWT JWTConfig
}

type JWTConfig struct {
    AccessSecret     string `env:"USER_AUTH_ACCESS_SECRET"`
    RefreshSecret    string `env:"USER_AUTH_REFRESH_SECRET"`
    AccessExpiresIn  int    `env:"USER_AUTH_ACCESS_EXPIRES_IN"`
    RefreshExpiresIn int    `env:"USER_AUTH_REFRESH_EXPIRES_IN"`
}

type AssetsConfig struct {
    FlagsPath string `env:"ASSETS_FLAGS_PATH"`
}
```

### Scraper Command Config

```go
type Config struct {
    config.DatabaseConfig
    config.LoggingConfig
    TED ScraperConfig
}

type ScraperConfig struct {
    BaseURL          string        `env:"TED_BASE_URL"`
    Timeout          time.Duration `env:"TED_TIMEOUT"`
    MaxRetries       int           `env:"TED_MAX_RETRIES"`
    RetryDelay       time.Duration `env:"TED_RETRY_DELAY"`
    UserAgent        string        `env:"TED_USER_AGENT"`
    MaxConcurrency   int           `env:"TED_MAX_CONCURRENCY"`
    DownloadDir      string        `env:"TED_DOWNLOAD_DIR"`
    CleanupAfter     time.Duration `env:"TED_CLEANUP_AFTER"`
    ScrapingInterval time.Duration `env:"TED_SCRAPING_INTERVAL"`
}
```

## Features

### Type Safety

The configuration loader uses Go generics to provide type-safe configuration loading. The function signature ensures that you get back exactly the type you expect.

### Environment Variable Priority

The configuration system loads values with explicit priority:
1. OS environment variables (system-level, highest priority)
2. .env file values (file-based, development defaults)

### Reflection-Based Parsing

The system uses reflection to automatically parse environment variables into struct fields based on the `env` tags.

### Supported Types

The configuration system supports the following Go types:
- `string`
- `int`, `int8`, `int16`, `int32`, `int64`
- `bool`
- `float32`, `float64`
- `time.Duration` (parsed from strings like "30s", "1h", "5m")

## Best Practices

1. **Use descriptive environment variable names**: Choose clear, descriptive names for your environment variables (e.g., `MONGODB_URI` instead of `DB_URI`).

2. **Use consistent naming conventions**: Use uppercase with underscores for environment variable names (e.g., `SERVER_HOST`, `LOG_LEVEL`).

3. **Group related variables**: Use prefixes to group related configuration (e.g., `MONGODB_*`, `REDIS_*`, `LOG_*`).

4. **Document your configuration**: Add comments in your `.env` files explaining what each variable does.

5. **Use appropriate types**: Use `time.Duration` for durations, proper numeric types for numbers, etc.

6. **Provide sensible defaults in .env**: Use `.env` files to provide development defaults and override with OS environment variables for production.

7. **Leverage the priority system**: Use `.env` for development defaults and OS environment variables for production overrides.

8. **Keep secrets in OS environment variables**: Never put sensitive values like API keys or passwords in `.env` files; use OS environment variables instead.

## Migration from YAML System

If you're migrating from the old YAML-based configuration system:

1. Remove `mapstructure` tags from your config structs
2. Add `env` tags to your config struct fields
3. Create `.env` files instead of `config.yaml` files
4. Update your `LoadConfig` calls to use the new system
5. Update function signatures to use the new config types

Example migration:

```go
// Old way (YAML)
type Config struct {
    Server config.ServerConfig `mapstructure:"server"`
    Database config.DatabaseConfig `mapstructure:"database"`
}

// New way (.env)
type Config struct {
    Server config.ServerConfig
    Database config.DatabaseConfig
}

// Old way (YAML loading)
conf, err := config.LoadConfig(".", &Config{})

// New way (.env loading) - same API!
conf, err := config.LoadConfig(".", &Config{})
```

## Environment Variable Naming Convention

The system follows these naming conventions:

- **Server**: `SERVER_*` (e.g., `SERVER_HOST`, `SERVER_PORT`)
- **Database**: `MONGODB_*` (e.g., `MONGODB_URI`, `MONGODB_NAME`)
- **Cache**: `REDIS_*` (e.g., `REDIS_HOST`, `REDIS_PORT`)
- **Logging**: `LOG_*` (e.g., `LOG_LEVEL`, `LOG_FORMAT`)
- **Rate Limiting**: `RATE_LIMIT_*` (e.g., `RATE_LIMIT_REQUESTS_PER_MINUTE`)

## Error Handling

The configuration system provides detailed error messages when:
- The `.env` file exists but cannot be loaded
- Environment variables cannot be parsed into the expected types
- Required fields are missing (handled by your application logic)

Note: Missing `.env` files do not cause errors - the system will continue with OS environment variables only.

## Security Considerations

1. **Never commit `.env` files with secrets**: Add `.env` to your `.gitignore` file and never include sensitive values
2. **Use OS environment variables for production secrets**: Set sensitive values like API keys and database passwords as OS environment variables
3. **Use environment-specific files**: Create different `.env` files for different environments (`.env.local`, `.env.development`, etc.)
4. **Validate sensitive values**: Always validate sensitive configuration values like API keys and database URIs
5. **Use secure defaults**: Provide secure default values for security-related configuration
6. **Leverage priority system for security**: Keep development defaults in `.env` and override with secure OS environment variables in production

## Deployment Strategies

### Development
```bash
# Use .env file for all configuration
./your-command
```

### Staging
```bash
# Use .env for defaults, override key values
export MONGODB_URI=mongodb://staging-db:27017
export LOG_LEVEL=warn
./your-command
```

### Production
```bash
# Override all critical values with OS environment variables
export MONGODB_URI=mongodb://prod-cluster:27017
export MONGODB_NAME=production
export LOG_LEVEL=error
export SERVER_HOST=0.0.0.0
./your-command
```

### Docker
```dockerfile
# Set production values in Dockerfile or docker-compose
ENV MONGODB_URI=mongodb://mongo:27017
ENV LOG_LEVEL=info
ENV SERVER_HOST=0.0.0.0
```