Skip to content

Block data import and export between .csv format and the most popular databases IBM DB2, Oracle, AWS AuroraDB, MongoDB, PostgreSQL, MySQL, SQLite3

License

Notifications You must be signed in to change notification settings

SSobol77/csv-db-sdk

Folders and files

NameName
Last commit message
Last commit date

Latest commit

Β 

History

75 Commits
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 
Β 

Repository files navigation

15_36_23

License: GPL v3 Python Versions Linux FreeBSD


πŸ“¦ CSV-DB-SDK

Universal Database Integration SDK for CSV Operations




Important

🚫 Ethical Restrictions

My works cannot be used in:

  • Military applications or systems
  • Surveillance technologies
  • Any activity violating human rights


πŸš€ Features

Category Details
Core Operations πŸ“₯ CSV-to-DB Import β€’ πŸ“€ DB-to-CSV Export β€’ πŸ”„ Bidirectional Sync
Database Support βœ… PostgreSQL β€’ βœ… MySQL β€’ βœ… SQLite β€’ πŸš€ MongoDB β€’ ☁️ AWS Aurora β€’ 🏒 Oracle β€’ πŸ–₯️ IBM DB2
SDK Advantages 🧩 Modular Design β€’ πŸ“š Client Libraries β€’ πŸ›  CLI Tools β€’ πŸ§ͺ Mock Testing Framework

⚑ Quick Start

Installation

# Clone repository
git clone https://github.com/SSobol77/csv-db-sdk.git
cd csv-db-sdk

# Basic Installation (Core Functionality)
pip install csv-db-sdk

# Development Setup (Testing + Coverage)
pip install "csv-db-sdk[testing]"

# Full Installation (All Features)
pip install "csv-db-sdk[full]"

Semantic Versioning Policy

Dependency Management

Strict SemVer compliance for all dependencies: psycopg2-binary ~= 2.9.9 # Compatible with 2.9.x (2.9.9 ≀ version < 3.0) boto3 ~= 1.34.112 # Compatible with 1.34.x (1.34.112 ≀ version < 2.0)

Version Guarantees
  • Major versions (X.0.0): Breaking API changes
  • Minor versions (1.X.0): Backwards-compatible features
  • Patch versions (1.0.X): Backwards-compatible bug fixes

Compatibility Matrix

Component Supported Versions Stability Level
PostgreSQL 12-16 Production
MySQL 5.7-8.1 Production
Oracle DB 19c-23c Verified
MongoDB 4.4-7.0 Production

Upgrade Recommendations

# Safe Upgrade Path
pip install --upgrade-strategy eager "csv-db-sdk>=1.2,<2.0"

# Version Pinning Example
echo "csv-db-sdk==1.2.3" >> production-requirements.txt

Basic Usage

from csv_db_sdk import PostgresConnector

# Initialize connector
config = {
    "host": "localhost",
    "user": "admin",
    "password": "secret",
    "database": "mydb"
}
pg = PostgresConnector(config)

# Import CSV to table
pg.import_csv("data/users.csv", "users_table")

# Export table to CSV
pg.export_csv("analytics/results.csv", "sales_data")


πŸ— Architecture

csv-db-sdk/
β”œβ”€β”€ πŸ“‚ core/               # SDK Core Components
β”‚   β”œβ”€β”€ connectors.py      # Base DB connector logic
β”‚   └── utilities.py       # CSV parsing/validation
β”œβ”€β”€ πŸ“‚ db_adapters/        # Database-specific implementations
β”‚   β”œβ”€β”€ postgres.py        # PostgreSQL adapter
β”‚   β”œβ”€β”€ mongodb.py         # MongoDB adapter
β”‚   └── ...                # Other databases
β”œβ”€β”€ πŸ“‚ examples/           # Ready-to-run scenarios
β”‚   β”œβ”€β”€ basic_import.py    # CSV β†’ DB example
β”‚   └── advanced_export.py # DB β†’ CSV with filtering
└── πŸ“‚ tests/              # Comprehensive test suite
    β”œβ”€β”€ unit/              # Isolated component tests
    └── integration/      # End-to-end workflow tests


πŸ”§ Database Configuration

Connection Templates

# PostgreSQL Example
postgres:
  host: "db-server.prod"
  port: 5432
  database: "analytics"
  user: "${DB_USER}"
  password: "${DB_PASS}"
  sslmode: "require"

# MongoDB Example
mongodb:
  uri: "mongodb+srv://cluster.prod.mongodb.net"
  authSource: "admin"
  tls: true

πŸ§ͺ Testing Strategy

Multi-level Validation:

# Run all tests
pytest tests/ -v

# Test specific database
pytest tests/postgres -v --cov=db_adapters.postgres

# Generate coverage report
pytest --cov-report html --cov=.
Test Type Coverage Tools Used
Unit Testing 92% Core logic pytest, unittest
Integration Tests 85% DB-specific workflows Docker, Testcontainers
Performance Bench 10k rows/sec (PostgreSQL) Locust, pyperf

🀝 Contributing

  1. Fork the repository
  2. Create feature branch: git checkout -b feat/amazing-feature
  3. Commit changes: git commit -m 'Add amazing feature'
  4. Push to branch: git push origin feat/amazing-feature
  5. Open Pull Request

πŸ“œ License

GNU GPLv3 - See LICENSE for full text.
πŸ“Œ Commercial use requires special permission - contact author for details.


πŸ“¬ Contact

Siergej Sobolewski
Email πŸš€
GitHub
LinkedIn

About

Block data import and export between .csv format and the most popular databases IBM DB2, Oracle, AWS AuroraDB, MongoDB, PostgreSQL, MySQL, SQLite3

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages