Skip to content

README.md

Latest commit

 

History

History
188 lines (166 loc) · 10.1 KB

README.md

File metadata and controls

188 lines (166 loc) · 10.1 KB
Raw

Kit4go

GitHub go.mod Go version Top Languages Go Report Card License Release CI Last Commit Tests codecov PR Sourcegraph Open Source Helpers TODOs

Common Go utility library for ad-tech, finance, and blockchain infrastructure.

Architecture

graph TB
    subgraph ROOT["github.com/v8fg/kit4go (root module — zero external deps)"]
        direction LR
        subgraph CONCURRENCY["Concurrency"]
            WP[workerpool]
            PL[pipeline]
            SEM[semaphore]
            RT[retry]
            WT[wtimer]
            DB[debounce]
            FO[fanout]
            SD[shutdown]
            BT[batcher]
        end
        subgraph ALGO["Algorithm Primitives"]
            BL[bloom]
            CM[countmin]
            HLL[hyperloglog]
            TK[topk]
            RS[reservoir]
            TR[trie]
            RB[ringbuffer]
            CH[consistenthash]
            LB[loadbalance]
        end
        subgraph CONTROL["Rate · Budget · Protection"]
            LM[limiter]
            BD[budget]
            HK[hotkey]
            FC[freqcap]
            ID[idempotency]
            BR[breaker]
        end
        subgraph CLIENTS["Clients"]
            HC[httpclient]
            TC[tcpclient]
            UC[udpclient]
        end
        subgraph SERVERS["Servers"]
            HS[httpserver]
            GS[grpcserver]
        end
        subgraph UTILS["Utilities"]
            BI[bit · str · datetime]
            NUM[number · random · otp]
            IP[ip · hash · base62]
            UI[uuid · json · config]
        end
    end

    subgraph SUBMODULES["Sub-modules (own go.mod — heavy deps isolated)"]
        direction LR
        L4G[log4go<br/>structured logger]
        KFK[kafka<br/>sarama/franz-go]
        PG[postgres<br/>pgx pool]
        RDS[redis<br/>go-redis]
        RDL[redislock<br/>distributed lock]
        RATE[rate<br/>Redis GCRA]
        GRPC[grpcclient<br/>retry + breaker]
        GRPS[grpcserver<br/>interceptors]
        EML[email<br/>go-mail SMTP]
        MET[metrics<br/>Prometheus]
        TRC[tracing<br/>OpenTelemetry]
    end

    HC -.-> BR
    HC -.-> RT
    GS -.-> SD
    HS -.-> SD
    L4G -.-> KFK
    RDL -.-> RDS
    RATE -.-> RDS
    GRPC -.-> BR

    style ROOT fill:#e8f5e9,stroke:#2e7d32
    style SUBMODULES fill:#fff3e0,stroke:#e65100
    style CONCURRENCY fill:#e3f2fd,stroke:#1565c0
    style ALGO fill:#f3e5f5,stroke:#6a1b9a
    style CONTROL fill:#fce4ec,stroke:#ad1457
    style CLIENTS fill:#e0f7fa,stroke:#006064
    style SERVERS fill:#e0f7fa,stroke:#006064
    style UTILS fill:#f5f5f5,stroke:#616161
Loading

Package list

Root module (github.com/v8fg/kit4go)

Category Packages
Concurrency workerpool · pipeline · semaphore · retry · wtimer · debounce · fanout · shutdown · batcher · backpressure · objpool · signalbus
Algorithms bloom · countmin · hyperloglog · topk · reservoir · trie · ringbuffer · consistenthash · loadbalance · priorityqueue · auction (2nd-price/multi-slot) · fsm
Rate & budget limiter (token-bucket/sliding-window/fixed-window/leaky/GCRA) · budget · rate (Redis-backed) · hotkey · freqcap · idempotency · breaker
Cache & storage cache (unified memory=lru/redis) · lru · shortlink
Finance money (ISO-4217 fixed-point) · decimal (fixed-point, math/big)
Clients httpclient · tcpclient · udpclient
Servers httpserver · grpcserver · middleware (request-id/ratelimit/CORS)
Observability latency (sharded tail-latency histogram)
Utilities bit · datetime · file · ip · json · number · str · uuid · xlo · random · otp · base62 · hash · config · maxprocs · backoff · health · stress · featureflag · errcode · hotreload · signing

Sub-modules (own go.mod — heavy deps isolated)

Module What Heavy deps
log4go async structured logging (console/file/kafka/net/io, sampling, ShardLogger, circuit breaker + spill failover, ~1M qps/core) sarama, sonic, goccy
kafka producer + consumer (sync/async, group, partition; sarama/franz-go unified) IBM/sarama
postgres pgx pool wrapper jackc/pgx/v5
clickhouse ClickHouse OLAP client wrapper (native protocol, PrepareBatch pass-through) ClickHouse/clickhouse-go
elasticsearch Elasticsearch search/analytics wrapper (Index/Get/Search/Delete, official go-elasticsearch/v8) elastic/go-elasticsearch
etcd etcd distributed-KV wrapper (service registration + discovery: KV/Lease/Watch) etcd-io/etcd
aerospike Aerospike high-throughput KV wrapper (Put/Get/Delete/BatchGet) aerospike/aerospike-client-go
minio S3/MinIO object-store client wrapper (Put/Get/Stat/Remove/List/Presign) minio/minio-go
mongo MongoDB document-store wrapper (Collection CRUD: Find/Insert/Update/Delete) mongodb/mongo-go-driver
redis Redis client wrapper redis/go-redis
redislock distributed lock (token-guarded Lua, auto-renew, onLost) redis/go-redis
rate Redis-backed GCRA rate limiter redis/go-redis
grpcclient gRPC client middleware (retry, breaker, metrics) grpc, protobuf
grpcserver gRPC server (interceptors, graceful shutdown) grpc, protobuf
email SMTP via go-mail (TLS Mandatory by default) wneessen/go-mail
metrics Prometheus wrapper prometheus/client_golang
tracing OpenTelemetry wrapper go.opentelemetry.io/otel
adaptive CPU-aware worker-pool (scales to keep CPU under a target, leaves headroom for latency-critical paths) shirou/gopsutil

Importing github.com/v8fg/kit4go/log4go does not pull pgx or grpc into your module graph — each sub-module owns only its own dependencies. Local development uses a committed go.work so go build/go test resolve all modules together.

Install

go get github.com/v8fg/kit4go                     # root utilities (60+ packages)
go get github.com/v8fg/kit4go/log4go              # structured logging (standalone)
go get github.com/v8fg/kit4go/kafka               # kafka producer/consumer (standalone)
go get github.com/v8fg/kit4go/postgres            # pgx pool (standalone)
go get github.com/v8fg/kit4go/redislock           # distributed lock (standalone)

Quality

  • Deep concurrency audit: 25+ rounds, ~40 real bugs fixed (deadlocks, races, leaks, panics, float-drift, buffer overflows). See QUALITY_RULES.md for the framework.
  • Fuzz testing: ~60 fuzz targets across all packages, catching edge cases unit tests miss.
  • log4go resilience: circuit breaker + spill failover, observable degradation, bounded shutdown. See log4go/RESILIENCE.md.
  • Callback-recover policy: library-owned workers recover panics (Recovered() + SetOnPanic).
  • CI: all 18 sub-modules, ubuntu + macOS, -race, -short.
  • Lint: golangci-lint v2 with 11 high-signal linters.
  • Coverage: 95%+ across all packages (most at 100%).
  • Security: govulncheck clean (go1.26.5). See CHANGELOG.md for CVE remediation.

Notes

If test failed, maybe effected by the inline, you can try: go test -v -gcflags=all=-l xxx_test.go.

Error-path tests use injectable interfaces (e.g. file.FS, otp.RandomReader, ip.AddrLookup, random.CryptoSource) with mockery mocks instead of runtime monkey-patching. Regenerate mocks with go generate ./... (mockery v2) after editing those interfaces. Each interface has a //go:generate mockery ... directive and a committed mock_*.go.

CMD

  • release check: make
  • coverage: make cover
  • format check: make fmt-check
  • format fixed: make fmt
  • misspell check: make misspell-check
  • golang lint: make golangci
  • escape analysis: make escape or ESCAPE_PATH=ip make escape