通用的go语言对象池:Go Commons Pool

jopen 6年前

Go commons pool是一个通用的go语言对象池,基于Java版本的Apache Commons Pool改写。Go commons pool实现了Java版本的主要功能,改写了大多数Java版本的测试用例,测试覆盖率达到90%,性能测试结果和Java版本的相近,已经可以用于生产环境,于是发布1.0版本

Go commons pool保留了Java版本的主要功能,包括:

  1. 自定义的 PooledObjectFactory.
  2. 丰富的设置选项,可以精确控制对象的生命周期。详细参看ObjectPoolConfig
    • 对象池是否是 LIFO (后进先出) 或者是 FIFO (先进先出)
    • 对象池的容量控制
    • 对象池对象的验证配置
    • 获取对象时是否阻塞以及最大等待时间配置
    • 对象池对象的回收机制配置(支持后台定时任务检测回收)
    • 对象池对象的抛弃机制配置(主要用于防止对象池对象借出后未归还,导致对象泄露)

但不包含以下Apache commons pool的功能:

  1. KeyedObjectPool 实现
  2. ProxiedObjectPool 实现
  3. 对象池的统计功能

Go commons pool 最主要的应用场景是各种连接池,当前Go下的各种缓存或数据库(比如redis/memcached)都自己实现了一个功能不太完备的连接池(比如缺少超时设置等),如果通过Go commons pool可以支持更丰富的设置。

Pool Configuration Option

Configuration option table, more detail description see ObjectPoolConfig

Option Default Description
Lifo true If pool is LIFO (last in, first out)
MaxTotal 8 The cap of pool
MaxIdle 8 Max "idle" instances in the pool
MinIdle 0 Min "idle" instances in the pool
TestOnCreate false Validate when object is created
TestOnBorrow false Validate when object is borrowed
TestOnReturn false Validate when object is returned
TestWhileIdle false Validate when object is idle, see TimeBetweenEvictionRunsMillis
BlockWhenExhausted true Whether to block when the pool is exhausted
MaxWaitMillis -1 Max block time, less than 0 mean indefinitely
MinEvictableIdleTimeMillis 1000 * 60 * 30 Eviction configuration,see DefaultEvictionPolicy
SoftMinEvictableIdleTimeMillis math.MaxInt64 Eviction configuration,see DefaultEvictionPolicy
NumTestsPerEvictionRun 3 The maximum number of objects to examine during each run evictor goroutine
TimeBetweenEvictionRunsMillis -1 The number of milliseconds to sleep between runs of the evictor goroutine, less than 0 mean not run

Usage

//use create func  pool := NewObjectPoolWithDefaultConfig(NewPooledObjectFactorySimple(          func() (interface{}, error) {              return &MyPoolObject{}, nil          }))  obj, _ := pool.BorrowObject()  pool.ReturnObject(obj)    //use custom Object factory    type MyObjectFactory struct {    }    func (this *MyObjectFactory) MakeObject() (*PooledObject, error) {      return NewPooledObject(&MyPoolObject{}), nil  }    func (this *MyObjectFactory) DestroyObject(object *PooledObject) error {      //do destroy      return nil  }    func (this *MyObjectFactory) ValidateObject(object *PooledObject) bool {      //do validate      return true  }    func (this *MyObjectFactory) ActivateObject(object *PooledObject) error {      //do activate      return nil  }    func (this *MyObjectFactory) PassivateObject(object *PooledObject) error {      //do passivate      return nil  }    pool := NewObjectPoolWithDefaultConfig(new(MyObjectFactory))  pool.Config.MaxTotal = 100  obj, _ := pool.BorrowObject()  pool.ReturnObject(obj)

more example please see pool_test.go and example_test.go

Note

PooledObjectFactory.MakeObject must return a pointer, not value. The following code will complain error.

pool := NewObjectPoolWithDefaultConfig(NewPooledObjectFactorySimple(      func() (interface{}, error) {          return "hello", nil      }))  obj, _ := pool.BorrowObject()  pool.ReturnObject(obj)

The right way is:

pool := NewObjectPoolWithDefaultConfig(NewPooledObjectFactorySimple(      func() (interface{}, error) {          var stringPointer = new(string)          *stringPointer = "hello"          return stringPointer, nil      }))

more example please see example_test.go

Dependency

PerformanceTest

The results of running the pool_perf_test is almost equal to the java version PerformanceTest

go test --perf=true

For Apache commons pool user

  • Direct use pool.Config.xxx to change pool config
  • Default config value is same as java version
  • If TimeBetweenEvictionRunsMillis changed after ObjectPool created, should call ObjectPool.StartEvictor to take effect. Java version do this on set method.
  • No KeyedObjectPool
  • No ProxiedObjectPool
  • No pool stats (TODO)

FAQ

FAQ

How to contribute

  • Choose one open issue you want to solve, if not create one and describe what you want to change.
  • Fork the repository on GitHub.
  • Write code to solve the issue.
  • Create PR and link to the issue.
  • Make sure test and coverage pass.
  • Wait maintainers to merge.

中文文档

License

Go Commons Pool is available under the Apache License, Version 2.0.


项目地址: https://github.com/jolestar/go-commons-pool