-
Notifications
You must be signed in to change notification settings - Fork 447
/
Copy pathChangeLog
239 lines (208 loc) · 9.17 KB
/
ChangeLog
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
2020-07-29 Alibaba Cloud.
* version: 2.4.8
* BUGFIX: fix bug that checkpoint.storage.db does not work since v2.4.6.
* BUGFIX: fix bug of orphan document filter core. #389.
* IMPROVE: remove listDatabase request when incr_sync.shard_key != auto.
#385.
* IMPROVE: add oplog_max_size and oplog_avg_size metric in restful: "repl/".
* BUGFIX: fix bug of checkpoint duplicate updating when HA switch.
* IMPROVE: increase syncer thread when fetching method is change stream.
* IMPROVE: add readConcern and writeConcern in client to solve the
orphan document problem reading from MongoS. #392
* IMPROVE: fix wrong name usage in mongoshake-stat script, thanks @Neal
Gosalia. #393.
2020-06-30 Alibaba Cloud.
* version: 2.4.7
* IMPROVE: add exit-point and safe shutdown mechanism. #375, #162
* IMPROVE: add UpdateLookup options in change stream. #350. thanks for
the PR by Raydy.
* IMPROVE: open full_sync.reader.read_document_count parameter to improve
full sync performance when meeting jumbo table.
* IMPROVE: polish log: remove function information in log.
2020-06-17 Alibaba Cloud.
* version: 2.4.6
* IMPROVE: when running full sync, change sync source from mongod to
mongos. #360
* IMPROVE: when source type is sharding, change checkpoint position from
config-server to mongos.
* BUGFIX: fix bug of whether the starting time oplog is legal: the oldest
oplog timestamp > checkpoint.start_position
* IMPROVE: convert kafka producer MaxMessageBytes configuration from 1MB
to 16MB. #369.
* IMPROVE: add target delay parameter just like MongoDB primary-secondary
SlaveDelay. #371.
2020-06-08 Alibaba Cloud.
* version: 2.4.5
* BUGFIX: fix bug of ignore error wrong when parsed error list is nil.
2020-06-05 Alibaba Cloud.
* version: 2.4.4
* BUGFIX: fix bug of "two $set" in update query. see #345.
* BUGFIX: fix bug of source mongodb judgement error. see #343.
* IMPROVE: solve the idempotency problem of oplog synchronization
during the initial sync period. see #345.
2020-05-14 Alibaba Cloud.
* version: 2.4.3
* BUGFIX: rename `full_sync.collection_exist_no_drop` to
`full_sync.collection_exist_drop`.
2020-04-22 Alibaba Cloud.
* version: 2.4.2
* BUGFIX: fix bug of applyOps run failed. #329
2020-04-09 Alibaba Cloud.
* version: 2.4.1
* IMPROVE: add speed limit policy in full sync stage.
* IMPROVE: modify speed limit policy to token bucket algorithm in
incremental sync.
* IMPROVE: add background index build policy, see full_sync.create_index. #316
* IMPROVE: add monitor in full sync stage.
* BUGFIX: fix bug of parsing oplog empty from kafka. #317
* BUGFIX: fix bug of checkpoint flush block. #323
2020-03-25 Alibaba Cloud.
* version: 2.4.0
* IMPROVE: support change stream to solve move chunk problem.
* IMRPOVE: add feature compatiable version(fcv) for configuration and
checkpoint.
* IMPROVE: optimize orphan document filter.
* IMPROVE: block password in conf and log.
* IMPROVE: retry to insert document when duplicate error happen in full
sync stage.
* IMRPOVE: add 'full_sync.create_index' in configuration.
* DISABLE: temporarily disable oplog stored to disk during document
replication released on v2.2.1.
2020-01-22 Alibaba Cloud.
* version: 2.2.1
* IMPROVE: support oplog stored to disk during document replication.
* IMPROVE: support reading from kafka in by different tunnel message type.
2019-11-20 Alibaba Cloud.
* version: 2.2.0
* this is a experimental version for sharding replication
* IMRPOVE: support filter orphan document in full synchronization for mongodb sharding
* IMPROVE: support move chunk in incremental synchronization for mongodb sharding
* IMPROVE: support ddl in incremental synchronization for mongodb sharding
* IMPROVE: support different sharding key or index at destination mongodb
* IMPROVE: support global consistence checkpoint
2019-09-30 Alibaba Cloud.
* version: 2.0.8
* BUGFIX: fetch data from kafka failed. update "sarama" driver, see #212.
* IMPROVE: support for the compilation of windows environment. see #233.
* IMPROVE: remove some parameters in configuration.
* IMRPOVE: add TPS in restful metric.
2019-08-17 Alibaba Cloud.
* version: 2.0.7
* BUGFIX: exit when the oldest ts bigger than full-sync-begin
timestamp so that data may lost between full sync and increase sync.
see #218.
* BUGFIX: bugfix: set white namespace filter for sharding replication
cause db filtered. see #222.
* IMPROVE: add switch in configuration to support syncing some special
database like 'admin', 'local', default is disbale. see #223.
* IMPROVE: set `adaptive.batching_max_size = 1024` by default to reduce
memory usage. see
https://github.com/alibaba/MongoShake/wiki/FAQ#q-mongoshake-crashed-because-of-oomout-of-memory-how-can-i-estimate-memory-usage
2019-08-08 Alibaba Cloud.
* version: 2.0.6
* IMPROVE: filter views in syncing, see #206.
* IMPROVE: when the synchronization mode is full synchronization,
support for the case where oplog does not exist. see #215.
* IMPROVE: let mongoshake exits when finish full syncing. see #210.
* IMPROVE: add `log.dir` to write log and pid file in absolute path.
see #211.
2019-07-29 Alibaba Cloud.
* version: 2.0.5
* IMPROVE: comparison.py support sharding comparison. see #197.
* BUGFIX: comparison.py fix bug of negative count. see #199.
* IMPROVE: set worker number to shards number. see #198.
* IMPROVE: support sharding active-active architecture. This is only
used in Alibaba-Cloud.
2019-07-15 Alibaba Cloud.
* version: 2.0.4
* BUG FIX: fix bug of batched oplog size exceeding 16MB, This bug was
not completely resolved in version v1.4.6. see #122 and #187.
* IMPROVE: add chinese comment in configuration.
* IMPROVE: print warning log when the oldest oplog is bigger than the
checkpoint timestamp, see #189.
* IMPROVE: flush heartbeat checkpoint with interval 180s. see #189.
* BUG FIX: use runCommand instead of applyOps to support sharding. see
#192.
* BUG FIX: partial log dump with parameter `all` equal to false.
2019-07-10 Alibaba Cloud.
* version: 2.0.3
* BUG FIX: update sentence incurs some fields lost.
* BUG FIX: modify start.sh
2019-07-09 Alibaba Cloud.
* version: 2.0.2
* BUG FIX: DDL createIndexes fail. Using `applyOps` in DDL replay. see
issue #176.
* BUG FIX: DDL filter applyOps command panic.
* IMPROVE: improve comparision script.
* IMRPOVE: update vinllen/mgo library to solve the problem that using too
many mgo connections.
2019-06-24 Alibaba Cloud.
* version: 2.0.1
* IMPROVE: add `mongo_connect_mode` in conf.
* IMPROVE: support writing into different mongodb when source mongo is
replica set.
* BUG FIX: optimize namespace filter for DDL.
2019-06-19 Alibaba Cloud.
* version: 2.0.0
* Feature: support full synchronization.
* Feature: support DDL synchronization including index and transaction.
* Feature: support namespace rename.
* BUG FIX: fix bug of dbref order.
2019-04-20 Alibaba Cloud.
* version: 1.5.0-unstable
* Feature: support full synchronization.
* Feature: support DDL synchronization including index and transaction.
2019-04-12 Alibaba Cloud.
* version: 1.4.6
* BUG FIX: fix bug of batched oplog size exceeding 16MB.
2018-12-20 Alibaba Cloud.
* version: 1.4.5
* IMPROVE: modify `build.sh` to solve the problem of building error when
download zip from github instead of `git clone`.
* IMPROVE: modify error log level of `_id` type unknown.
2018-11-27 Alibaba Cloud.
* version: 1.4.4
* BUG FIX: enable bulk.Unordered when using bulk writer, so all the
operations will be executed even some of them have failed. At the
mean time, disable `replayer.executor.upsert` and
`replayer.executor.insert_on_dup_update` by default.
* IMPROVE: filter config database.
* IMPROVE: print `_id` type and value when can't be hashed.
* IMPROVE: disbale print version in command.
* IMPROVE: add `adaptive.batching_max_size` and `fetcher.buffer_capacity`
in configuration so users can control the memory usage.
* BUG FIX: do not use bulk writer to operate index.
2018-09-25 Alibaba Cloud.
* version: 1.4.3
* IMPROVE: stop syncing data when collection capped error happen in source
database.
2018-09-19 Alibaba Cloud.
* version: 1.4.2
* BUG FIX: fix bug of `start.sh` script commited by `ManleyLiu`.
* BUG FIX: fix bug of error judge sentence in bulk writer commited by
`chensi01`.
* IMPROVE: add timeout configuration and flush strategy for the oplog
reader .
* IMPROVE: remove "send [0] logs" hint in log.
2018-09-04 Alibaba Cloud.
* version: 1.4.1
* BUG FIX: fix bug of the time zone converter.
2018-09-03 Alibaba Cloud.
* version: 1.4.0
* Feature: Add receiver which is used to connect to different tunnels
like rpc, tcp, file, mock and kafka.
2018-08-08 Alibaba Cloud.
* version: 1.2.1
* BUG FIX: remove `$v` field in oplog to fix the bug in MongoDB 3.6.
2018-07-29 Alibaba Cloud.
* version: 1.2.0
* Feature: Use general write instead of op_command. When the version is
greater than or equal to 3.2, the bulk writer is used for batch writing,
otherwise the single writer is used for single writing.
2018-07-18 Alibaba Cloud.
* version: 1.0.0
* BUG FIX: upsert/remove error when old value isn't exist.
* BUG FIX: connect to Alibaba Cloud MongoDB instance without permission.
* BUG FIX: some logs can't be print out when crash.
2018-06-28 Alibaba Cloud.
* mongo-shake: initial release.