|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 |
db1库下的hoo集合使用hash分片，通过手工拆分shrs02的chunk触发moveChunk 可以观察到shrs02的oplog执行了delete操作{"op":"d"} shrs02:PRIMARY> db.oplog.rs.find({"ns" : "db1.hoo"}).sort({$natural:-1}).limit(5); { "ts" : Timestamp(1550548015, 279), "t" : NumberLong(1), "h" : NumberLong("-304152233719500506"), "v" : 2, "op" : "d", "ns" : "db1.hoo", "ui" : UUID("8d7b3877-cc2e-4e86-9a45-e66061e58697"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T03:46:55.082Z"), "o" : { "x" : 2, "_id" : ObjectId("5c6b76c0e5c6c8a235bca558") } } { "ts" : Timestamp(1550548015, 278), "t" : NumberLong(1), "h" : NumberLong("-5045599070557482756"), "v" : 2, "op" : "d", "ns" : "db1.hoo", "ui" : UUID("8d7b3877-cc2e-4e86-9a45-e66061e58697"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T03:46:55.082Z"), "o" : { "x" : 2, "_id" : ObjectId("5c6b76c0e5c6c8a235bca556") } } { "ts" : Timestamp(1550548015, 277), "t" : NumberLong(1), "h" : NumberLong("5120191885956451298"), "v" : 2, "op" : "d", "ns" : "db1.hoo", "ui" : UUID("8d7b3877-cc2e-4e86-9a45-e66061e58697"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T03:46:55.082Z"), "o" : { "x" : 2, "_id" : ObjectId("5c6b76c0e5c6c8a235bca543") } } { "ts" : Timestamp(1550548015, 276), "t" : NumberLong(1), "h" : NumberLong("4061724698943275872"), "v" : 2, "op" : "d", "ns" : "db1.hoo", "ui" : UUID("8d7b3877-cc2e-4e86-9a45-e66061e58697"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T03:46:55.082Z"), "o" : { "x" : 2, "_id" : ObjectId("5c6b76c0e5c6c8a235bca53c") } } { "ts" : Timestamp(1550548015, 275), "t" : NumberLong(1), "h" : NumberLong("-4863719428805872629"), "v" : 2, "op" : "d", "ns" : "db1.hoo", "ui" : UUID("8d7b3877-cc2e-4e86-9a45-e66061e58697"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T03:46:55.082Z"), "o" : { "x" : 2, "_id" : ObjectId("5c6b76c0e5c6c8a235bca538") } } 同样可以看到shrs01的oplog记录着insert操作{"op":"i"} shrs01:PRIMARY> use local switched to db local shrs01:PRIMARY> db.oplog.rs.find({"ns" : "db1.hoo"}).sort({$natural:-1}).limit(5); { "ts" : Timestamp(1550547112, 11052), "t" : NumberLong(1), "h" : NumberLong("-305777054114754335"), "v" : 2, "op" : "i", "ns" : "db1.hoo", "ui" : UUID("8d7b3877-cc2e-4e86-9a45-e66061e58697"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T03:31:52.515Z"), "o" : { "_id" : ObjectId("5c6b76c0e5c6c8a235bca558"), "x" : 2, "udt" : ISODate("2019-02-19T03:23:44.990Z"), "type" : 0, "type2" : 1 } } { "ts" : Timestamp(1550547112, 11051), "t" : NumberLong(1), "h" : NumberLong("669359981315100370"), "v" : 2, "op" : "i", "ns" : "db1.hoo", "ui" : UUID("8d7b3877-cc2e-4e86-9a45-e66061e58697"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T03:31:52.515Z"), "o" : { "_id" : ObjectId("5c6b76c0e5c6c8a235bca556"), "x" : 2, "udt" : ISODate("2019-02-19T03:23:44.987Z"), "type" : 2, "type2" : 1 } } { "ts" : Timestamp(1550547112, 11050), "t" : NumberLong(1), "h" : NumberLong("-4175246498915868787"), "v" : 2, "op" : "i", "ns" : "db1.hoo", "ui" : UUID("8d7b3877-cc2e-4e86-9a45-e66061e58697"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T03:31:52.515Z"), "o" : { "_id" : ObjectId("5c6b76c0e5c6c8a235bca543"), "x" : 2, "udt" : ISODate("2019-02-19T03:23:44.958Z"), "type" : 2, "type2" : 0 } } { "ts" : Timestamp(1550547112, 11049), "t" : NumberLong(1), "h" : NumberLong("8144974434003347592"), "v" : 2, "op" : "i", "ns" : "db1.hoo", "ui" : UUID("8d7b3877-cc2e-4e86-9a45-e66061e58697"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T03:31:52.515Z"), "o" : { "_id" : ObjectId("5c6b76c0e5c6c8a235bca53c"), "x" : 2, "udt" : ISODate("2019-02-19T03:23:44.946Z"), "type" : 6, "type2" : 4 } } { "ts" : Timestamp(1550547112, 11048), "t" : NumberLong(1), "h" : NumberLong("-1340638049932755147"), "v" : 2, "op" : "i", "ns" : "db1.hoo", "ui" : UUID("8d7b3877-cc2e-4e86-9a45-e66061e58697"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T03:31:52.515Z"), "o" : { "_id" : ObjectId("5c6b76c0e5c6c8a235bca538"), "x" : 2, "udt" : ISODate("2019-02-19T03:23:44.938Z"), "type" : 4, "type2" : 6 } } db1库下的t_user集合使用range分片，通过手工拆分shrs02的chunk触发moveChunk 可以观察到shrs01的oplog也是记录insert操作{"op":"i"} mongos> sh.splitFind("db1.t_user", {userid:50000}); shrs01:PRIMARY> use local switched to db local shrs01:PRIMARY> db.oplog.rs.find({"ns" : "db1.t_user"}).sort({$natural:-1}).limit(5); { "ts" : Timestamp(1550544266, 23696), "t" : NumberLong(1), "h" : NumberLong("6640796598824740378"), "v" : 2, "op" : "i", "ns" : "db1.t_user", "ui" : UUID("25190278-5b47-4c80-8138-a832f40840cb"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T02:44:26.617Z"), "o" : { "_id" : ObjectId("5c6b6c54806fa64b3bd55089"), "userid" : 50000, "name" : "user50000", "age" : 25 } } { "ts" : Timestamp(1550544266, 23695), "t" : NumberLong(1), "h" : NumberLong("5097978940295012072"), "v" : 2, "op" : "i", "ns" : "db1.t_user", "ui" : UUID("25190278-5b47-4c80-8138-a832f40840cb"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T02:44:26.617Z"), "o" : { "_id" : ObjectId("5c6b6c54806fa64b3bd55088"), "userid" : 49999, "name" : "user49999", "age" : 25 } } { "ts" : Timestamp(1550544266, 23694), "t" : NumberLong(1), "h" : NumberLong("-9206589727349063515"), "v" : 2, "op" : "i", "ns" : "db1.t_user", "ui" : UUID("25190278-5b47-4c80-8138-a832f40840cb"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T02:44:26.617Z"), "o" : { "_id" : ObjectId("5c6b6c54806fa64b3bd55087"), "userid" : 49998, "name" : "user49998", "age" : 25 } } { "ts" : Timestamp(1550544266, 23693), "t" : NumberLong(1), "h" : NumberLong("-5836469842737896486"), "v" : 2, "op" : "i", "ns" : "db1.t_user", "ui" : UUID("25190278-5b47-4c80-8138-a832f40840cb"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T02:44:26.617Z"), "o" : { "_id" : ObjectId("5c6b6c54806fa64b3bd55086"), "userid" : 49997, "name" : "user49997", "age" : 25 } } { "ts" : Timestamp(1550544266, 23692), "t" : NumberLong(1), "h" : NumberLong("6973611600053813704"), "v" : 2, "op" : "i", "ns" : "db1.t_user", "ui" : UUID("25190278-5b47-4c80-8138-a832f40840cb"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T02:44:26.617Z"), "o" : { "_id" : ObjectId("5c6b6c54806fa64b3bd55085"), "userid" : 49996, "name" : "user49996", "age" : 25 } } 可以观察到shrs02的oplog也是记录insert操作{"op":"d"} shrs02:PRIMARY> use local switched to db local shrs02:PRIMARY> db.oplog.rs.find({"ns" : "db1.t_user"}).sort({$natural:-1}).limit(5); { "ts" : Timestamp(1550545177, 464), "t" : NumberLong(1), "h" : NumberLong("6274443387502896231"), "v" : 2, "op" : "d", "ns" : "db1.t_user", "ui" : UUID("25190278-5b47-4c80-8138-a832f40840cb"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T02:59:37.085Z"), "o" : { "userid" : 50000, "_id" : ObjectId("5c6b6c54806fa64b3bd55089") } } { "ts" : Timestamp(1550545177, 463), "t" : NumberLong(1), "h" : NumberLong("4608755019998843904"), "v" : 2, "op" : "d", "ns" : "db1.t_user", "ui" : UUID("25190278-5b47-4c80-8138-a832f40840cb"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T02:59:37.085Z"), "o" : { "userid" : 49999, "_id" : ObjectId("5c6b6c54806fa64b3bd55088") } } { "ts" : Timestamp(1550545177, 462), "t" : NumberLong(1), "h" : NumberLong("-1809926020796113867"), "v" : 2, "op" : "d", "ns" : "db1.t_user", "ui" : UUID("25190278-5b47-4c80-8138-a832f40840cb"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T02:59:37.085Z"), "o" : { "userid" : 49998, "_id" : ObjectId("5c6b6c54806fa64b3bd55087") } } { "ts" : Timestamp(1550545177, 461), "t" : NumberLong(1), "h" : NumberLong("-4040712242599584545"), "v" : 2, "op" : "d", "ns" : "db1.t_user", "ui" : UUID("25190278-5b47-4c80-8138-a832f40840cb"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T02:59:37.085Z"), "o" : { "userid" : 49997, "_id" : ObjectId("5c6b6c54806fa64b3bd55086") } } { "ts" : Timestamp(1550545177, 460), "t" : NumberLong(1), "h" : NumberLong("3894943139412829378"), "v" : 2, "op" : "d", "ns" : "db1.t_user", "ui" : UUID("25190278-5b47-4c80-8138-a832f40840cb"), "fromMigrate" : true, "wall" : ISODate("2019-02-19T02:59:37.085Z"), "o" : { "userid" : 49996, "_id" : ObjectId("5c6b6c54806fa64b3bd55085") } } 从日志中可以看出chunk迁移在源分片上表现为删除操作，在目标分片上表现为插入操作 mongos> sh.status() #最终的分片集群信息 --- Sharding Status --- sharding version: { "_id" : 1, "minCompatibleVersion" : 5, "currentVersion" : 6, "clusterId" : ObjectId("5c6b6a17e5837b6787ff6a09") } shards: { "_id" : "shrs01", "host" : "shrs01/10.11.12.140:7101,10.11.12.140:7102,10.11.12.140:7103", "state" : 1 } { "_id" : "shrs02", "host" : "shrs02/10.11.12.140:7201,10.11.12.140:7202,10.11.12.140:7203", "state" : 1 } active mongoses: "3.6.10" : 1 autosplit: Currently enabled: yes balancer: Currently enabled: yes Currently running: no Failed balancer rounds in last 5 attempts: 0 Migration Results for the last 24 hours: 3 : Success databases: { "_id" : "config", "primary" : "config", "partitioned" : true } config.system.sessions shard key: { "_id" : 1 } unique: false balancing: true chunks: shrs01 1 { "_id" : { "$minKey" : 1 } } -->> { "_id" : { "$maxKey" : 1 } } on : shrs01 Timestamp(1, 0) { "_id" : "db1", "primary" : "shrs02", "partitioned" : true } db1.hoo shard key: { "x" : "hashed" } unique: false balancing: true chunks: shrs01 3 shrs02 3 { "x" : { "$minKey" : 1 } } -->> { "x" : NumberLong("-4611686018427387902") } on : shrs01 Timestamp(2, 2) { "x" : NumberLong("-4611686018427387902") } -->> { "x" : NumberLong(0) } on : shrs01 Timestamp(2, 3) { "x" : NumberLong(0) } -->> { "x" : NumberLong("2433984061343616404") } on : shrs01 Timestamp(3, 0) { "x" : NumberLong("2433984061343616404") } -->> { "x" : NumberLong("4470791281878691347") } on : shrs02 Timestamp(3, 1) { "x" : NumberLong("4470791281878691347") } -->> { "x" : NumberLong("4611686018427387902") } on : shrs02 Timestamp(2, 9) { "x" : NumberLong("4611686018427387902") } -->> { "x" : { "$maxKey" : 1 } } on : shrs02 Timestamp(2, 5) db1.t_user shard key: { "userid" : 1 } unique: true balancing: true chunks: shrs01 1 shrs02 1 { "userid" : { "$minKey" : 1 } } -->> { "userid" : 50001 } on : shrs01 Timestamp(2, 0) { "userid" : 50001 } -->> { "userid" : { "$maxKey" : 1 } } on : shrs02 Timestamp(2, 1) 总结： (1) mongos分片集群最小的逻辑存储单元（默认64M），表示自身所在分片下一个分片key值的连续范围 (2) chunk迁移在shard之间的oplog记录的是普通的插入删除操作 (3) chunk迁移完成以后才更新config server的元数据，最后才异步删除源shard的数据 |
暂无评论