• 1. Redis NoSQL系统@朱青玲 2012-12-19 WED
  • 2. AgendaRedis ServiceStack.Redis ServiceStack.Redis示例 SellerCube系统规范 Demo 参考资料
  • 3. 介绍Redis is an open source, advanced key-value store. It is often referred to as a data structure server since keys can contain strings, hashes, lists, sets and sorted sets. Redis是一款开源的高级键值存储器。它通常被称之为数据结构服务器,因为键可以包含字符串, 哈希, 列表, 集合和有序集合。
  • 4. 数据类型字符串 string 哈希表 hash 列表 list 集合 set (不允许重复) 有序集合 sorted set
  • 5. 特点与Memcached一样使用内存存储 支持VM虚拟内存机制,不受物理内存大小限制 支持持久化(snapshot/aof) 支持事务 丰富的数据类型 更丰富的操作,如push/pop、add/remove及取交集并集和差集等 支持多数据库切换 支持master-slave 主从同步,与mysql相似
  • 6. 性能测试结果基于: 50个并发执行100,000个请求 设置和获取一个256字节字符串 Linux 2.6 Xeon X3320 2.5GHz 本机测试 127.0.0.1 写的速度是110,000次/s 读的速度是81,000次/s 数据来源:http://redis.io/topics/benchmarks
  • 7. AgendaRedis ServiceStack.Redis ServiceStack.Redis示例 SellerCube系统规范 Demo 参考资料
  • 8. ServiceStack.RedisRedis C#接口,前身redis-sharp 面向接口编程,易接入IoC/DI 支持基于文本类型操作 IRedisClient,以及POCO类型 IRedisTypedClient 支持负载均衡的连接池管理 BasicRedisClientManager(本机),PooledRedisClientManager(跨网络) 类层次:IRedisTypedClient (POCO) > IRedisClient (string) > IRedisNativeClient (raw byte[]) 序列化通过ServiceStack.Text实现
  • 9. ServiceStack.Redis主要接口基于文本操作 IRedisClient IRedisList IRedisSet IRedisSortedSet IRedisHash 基于POCO操作 IRedisTypedClient IRedisList IRedisSet IRedisSortedSet IRedisHash
  • 10. ServiceStack.Redis主要接口事务 IRedisTransaction IRedisQueueOperation 缓存 ICacheClient Pub/Sub 消息订阅与分发 IRedisSubscription
  • 11. AgendaRedis ServiceStack.Redis ServiceStack.Redis示例 SellerCube系统规范 Demo 参考资料
  • 12. ServiceStack.Redis 示例(1)使用 PooledRedisClientManager连接池 static PooledRedisClientManager pooledClientManager; if (null == pooledClientManager) { // 支持多个服务端实现负载均衡 string redisServerHosts = "192.168.1.80:6379,192.168.1.81:6380"; pooledClientManager = new PooledRedisClientManager(redisServerHosts); } // 得到IRedisClient var client = pooledClientManager.GetClient();
  • 13. ServiceStack.Redis 示例(2)使用 IRedisClient using (redisClient) { string key = "test_key"; // ok 返回是否成功 bool ok = redisClient.SetEntry(key, "value from redis client of " + key); string retData = redisClient.GetValue(key); // 结果为value from redis client of test_key } /* 索引访问法 */ redisClient[key] = "value from redis client of " + key; string retData = redisClient[key];
  • 14. ServiceStack.Redis 示例(2)使用 IRedisClient /* 各类型获取方法 */ // 获取列表类型 IRedisList key2List = redisClient.Lists[key2]; // 获取集合类型 IRedisSet key2Set = redisClient.Sets[key2]; // 获取有序集合类型 IRedisSortedSet key2SortedSet = redisClient.SortedSets[key2]; // 获取Hash类型,IDictionary IRedisHash key2Hash = redisClient.Hashes[key2];
  • 15. ServiceStack.Redis 示例(3)使用 IRedisTypedClient using (redisClient) // redisClient is IRedisClient using (IRedisTypedClient typedClient = redisClient.GetTypedClient())// 需指定泛型类型 { typedClient.SetEntry( key, new Product(){ Id=typedClient.GetNextSequence(), // 获取递增序号,必须为IRedisTypedClient Name="Product ABC" }); Product retData2 = typedClient.GetValue(key); Console.WriteLine(retData2.Name); }
  • 16. ServiceStack.Redis 示例(3)使用 IRedisTypedClient /* 索引访问法,与IRedisClient相同 */ typedClient[key] = new Product(){ Id=typedClient.GetNextSequence(), Name="Product ABC" }; Product retData = typedClient[key];
  • 17. ServiceStack.Redis 示例(3)使用 IRedisTypedClient /* 各类型获取方法 */ // 获取列表类型 IRedisList key1List = typedClient.Lists[key1]; // 获取集合类型 IRedisSet key1Set = typedClient.Sets[key1]; // 获取有序集合类型IRedisSortedSet key1SortedSet = typedClient.SortedSets[key1]; // 获取Hash类型 IRedisHash key1Hash = typedClient.GetHash(key1); // 获取IRedisClient IRedisClient originalRedisClient = typedClient.RedisClient;
  • 18. ServiceStack.Redis 示例(3)使用 IRedisTypedClient /* 数据库式访问方法 */ public List GetAnswersForQuestion(long questionId) { using (var redis = RedisManager.GetClient()) // IRedisClient { return redis.As().GetRelatedEntities(questionId); } } // 定义: IRedisTypedClient IRedisClient.As() // 定义: List IRedisTypedClient.GetRelatedEntities(object parentId) // 分解 IRedisTypedClient typedClient = redis.As(); return typedClient.GetRelatedEntities(questionId);
  • 19. ServiceStack.Redis 示例(3)public void StoreQuestion(Question question) { using (var redis = RedisManager.GetClient()) { var redisQuestions = redis.As(); // IRedisTypedClient if (question.Tags == null) question.Tags = new List(); if (question.Id == default(long)) { question.Id = redisQuestions.GetNextSequence(); question.CreatedDate = DateTime.UtcNow; //Increment the popularity for each new question tag question.Tags.ForEach(tag => redis.IncrementItemInSortedSet(TagIndex.All, tag, 1)); } redisQuestions.Store(question); // 保存 redisQuestions.AddToRecentsList(question); // 保存到最近列表 } }
  • 20. ServiceStack.Redis 示例(4)使用 ICachedClient 不需直接操作ICacheClient,只需使用IRedisClient/IRedisTypedClient,两者都已实现ICacheClient // redisClient可以为IRedisClient或IRedisTypedClient类型 using (redisClient) { string key = "test_cache_key"; bool ok = redisClient.SetEntry(key, "value from cached redis client of " + key, TimeSpan.FromSeconds(5)); // // 第三个参数为过期时间 string retData = redisClient.GetValue(key); // 结果为value from cached redis client of test_cache_key Thread.Sleep(5000); retData = redisClient.GetValue(key); // 结果为空 }
  • 21. ServiceStack.Redis 示例(5)并发锁 // 支持IRedisTypedClient和IRedisClient using (redisClient.AcquireLock("testlock")) // IRedisClient // using (redisClient.AcquireLock()) // IRedisTypedClient { Console.WriteLine("client {0} acquired lock", clientNo); var counter = redisClient.Get("atomic-counter"); //Add an artificial delay to demonstrate locking behaviour Thread.Sleep(100); redisClient.Set("atomic-counter", counter + 1); Console.WriteLine("client {0} released lock", clientNo); }
  • 22. ServiceStack.Redis 示例(5)并发锁 /* Output: */ client 1 acquired lock client 1 released lock client 3 acquired lock client 3 released lock client 4 acquired lock client 4 released lock client 5 acquired lock client 5 released lock client 2 acquired lock client 2 released lock atomic-counter after 1sec: 5
  • 23. ServiceStack.Redis 示例(6)事务 int callbackResult; // 支持IRedisClient/IRedisTypedClient using (var trans = redis.CreateTransaction()) { trans.QueueCommand(r => r.Increment("key")); // Increment为增一操作 trans.QueueCommand(r => r.Increment("key"), i => callbackResult = i); // 增一结果返回保存到i/callbackResult trans.Commit(); // 提交事务 }
  • 24. ServiceStack.Redis 示例(7) 消息订阅与分发/* 订阅方 */ const int noOfClients = 2; for (var i = 1; i <= noOfClients; i++) { var clientNo = i; ThreadPool.QueueUserWorkItem(x => { using (var redisConsumer = new RedisClient(TestConfig.SingleHost)) using (var subscription = redisConsumer.CreateSubscription()) { var messagesReceived = 0; // OnSubscribe 事件,在订阅时触发 subscription.OnSubscribe = channel => { Console.WriteLine("Client #{0} Subscribed to '{1}'", clientNo, channel); };
  • 25. ServiceStack.Redis 示例(7) 消息订阅与分发 // OnUnSubscribe 事件,在取消订阅时触发 subscription.OnUnSubscribe = channel => { Console.WriteLine("Client #{0} UnSubscribed from '{1}'", clientNo, channel); };
  • 26. ServiceStack.Redis 示例(7) 消息订阅与分发 // OnMessage 事件,在收到消息时触发 subscription.OnMessage = (channel, msg) => { Console.WriteLine("Client #{0} Received '{1}' from channel '{2}'", clientNo, msg, channel); // 当消息数=3时,取消订阅 if (++messagesReceived == PublishMessageCount) { subscription.UnSubscribeFromAllChannels(); } }; Console.WriteLine("Client #{0} started Listening On '{1}'", clientNo, ChannelName); subscription.SubscribeToChannels(ChannelName); // 订阅,阻塞模式 } Console.WriteLine("Client #{0} EOF", clientNo); }); }
  • 27. ServiceStack.Redis 示例(7) 消息订阅与分发/* 分发方 */ using (var redisClient = new RedisClient(TestConfig.SingleHost)) { for (var i = 1; i <= PublishMessageCount; i++) { var message = MessagePrefix + i; // 发布消息 redisClient.PublishMessage(ChannelName, message); } }
  • 28. ServiceStack.Redis 示例(7) 消息订阅与分发输出 Client #1 started Listening On 'CHANNEL' Client #2 started Listening On 'CHANNEL' Client #1 Subscribed to 'CHANNEL' Client #2 Subscribed to 'CHANNEL' Publishing 'MESSAGE 1' to 'CHANNEL' Client #1 Received 'MESSAGE 1' from channel 'CHANNEL' Client #2 Received 'MESSAGE 1' from channel 'CHANNEL' Publishing 'MESSAGE 2' to 'CHANNEL' Client #1 Received 'MESSAGE 2' from channel 'CHANNEL' Client #2 Received 'MESSAGE 2' from channel 'CHANNEL' Publishing 'MESSAGE 3' to 'CHANNEL' Client #1 Received 'MESSAGE 3' from channel 'CHANNEL' Client #2 Received 'MESSAGE 3' from channel 'CHANNEL' Client #1 UnSubscribed from 'CHANNEL' Client #1 EOF Client #2 UnSubscribed from 'CHANNEL' Client #2 EOF
  • 29. ServiceStack.Redis 示例(8)其它IRedisTypedClient功能 获取递增序号 `long sequence = typedClient.GetNextSequence();` 复位序号 `typedClient.SetSequence(0);` 加入最近列表 void AddToRecentsList(T value) 从最近列表获取最近数据 List GetLatestFromRecentsList(int skip, int take) 从最近列表获取最早数据 List GetEarliestFromRecentsList(int skip, int take)
  • 30. AgendaRedis ServiceStack.Redis ServiceStack.Redis示例 SellerCube系统规范 Demo 参考资料
  • 31. SellerCube 系统规范缓存键生成规则 // 生成键方法 string SCBRedisKey.GenerateKey(KeyType keyType, string className, string methodName, params string[] arguments) 参数: KeyType 键类型,可选值 Cache // 用作缓存 Database, // 用作数据库 className 类型名称 methodName 方法名称 arguments 扩展参数,可选 返回: 示例:scb:cache:className:methodName:arg1:arg2:arg3
  • 32. SellerCube 系统规范 数据存取方法使用Redis作用定时过期缓存 using SellerCube.Core; using ServiceStack.Redis; // 连接故障或错误时,返回null IRedisClient redisClient = SCBRedis.GetClient(); // 必须使用using包裹,才能自动释放连接数 using (redisClient) { // 生成缓存键 string cacheKey = SCBRedisKey.GenerateKey(SCBRedisKey.KeyType.Cache, "SmartSeller", "get_related_sku_listing", eBayUserID, site, sku); // 先判断redisClient是否连接正常,以便处理连接中断故障 var cachedValue = null != redisClient ?redisClient.Get>(cacheKey) :null;
  • 33. SellerCube 系统规范 数据存取方法 if (null == cachedValue) // 当无数据时,需访问数据库 { string sql = string.Format(" SELECT * FROM dbo.fun_GetRelatedSKUListing('{0}','{1}','{2}') ",eBayUserID, site, sku); IQuery query = GetSQLQuery(sql).SetResultTransformer(Transformers.AliasToEntityMap); cachedValue = query.List(); // 写入缓存Redis, 1小时超时 if(null != redisClient) // 容错处理 redisClient.Set>( cacheKey, cachedValue,TimeSpan.FromHours(1.0)); } return cachedValue; } 注意:Hashtable类型要求所有字段类型必须一致,否则会报InvalidCastException类型转换错误
  • 34. SellerCube 系统规范 数据存取方法持久化数据仓库 using SellerCube.Core; using ServiceStack.Redis; public BaseMenuTreeService(bool updateCache) { if (!updateCache) { // 使用Redis读取菜单树 IRedisClient redisClient = SCBRedis.GetClient(); using (redisClient) { string serializedString = redisClient.Get(CacheKey); if (!string.IsNullOrEmpty(serializedString)) // 先判断是否存在数据,才执行反序列化操作 _menu = serializedString.DeserializeObject(); else _menu = null; }
  • 35. SellerCube 系统规范 数据存取方法 if (null != _menu) return; // 无数据时自动进入生成缓存步骤 } _menu = new BaseMenuTree(0); buildMenu(ref _menu,1); WriteCache(); }
  • 36. SellerCube 系统规范 数据存取方法public void WriteCache() { // 使用Redis保存菜单树 IRedisClient redisClient = SCBRedis.GetClient(); using (redisClient) { string serializedString = _menu.SerializeObject(); redisClient.Set(CacheKey, serializedString); // 未指定过期时间,为永久有效 } } ServiceStack.Text不能处理树状结构的序列化,需要使用SCB扩展函数执行序列化工作 string SellerCube.Core.SCBSerialization.SerializeObject(this object) T SellerCube.Core.SCBSerialization.DeserializeObject(this string)
  • 37. AgendaRedis ServiceStack.Redis ServiceStack.Redis示例 SellerCube系统规范 Demo 参考资料
  • 38. DEMOTelnet连接与操作 C# 示例 缓存 事务 IRedisTypedClient POCO类型操作 Sub/Pub 消息订阅分发 本地Redis服务器测试环境 192.168.1.80:6379 示例代码位于:^/192.168.1.1/共享文件/Playground/RedisDemo
  • 39. 参考资料官网 http://redis.io 命令列表 http://redis.io/commands Redis中文站 http://redis.cn/commands.html 命令中文参考 http://redis.readthedocs.org/en/latest/ ServiceStack.Redis网址 https://github.com/ServiceStack/ServiceStack.Redis ServiceStack.Redis类结构图 http://servicestack.net/img/Redis-annotated.png 《Redis学习笔记》 Redis学习笔记.pdf Redis性能测试与说明 http://redis.io/topics/benchmarks
  • 40. 谢谢 ^/SVN/文档/C 技术交流