Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix some load bugs #2384

Merged
merged 11 commits into from
Dec 5, 2019
Merged

Fix some load bugs #2384

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
42ab7d0
Optimize some routine load job's processing logic
morningman Dec 4, 2019
67c61e9
change to master daemon
morningman Dec 4, 2019
9991811
fix compile bug
morningman Dec 4, 2019
4e031bc
rebase master
morningman Dec 4, 2019
b863862
fix bug
morningman Dec 4, 2019
249539f
fix bug2
morningman Dec 4, 2019
0149ef2
fix bug
morningman Dec 4, 2019
a264cb2
only choose on backend
morningman Dec 4, 2019
b29a47f
fix bug
morningman Dec 5, 2019
2e61cec
remove routine load task begin limit check
morningman Dec 5, 2019
a040094
change ut http port
morningman Dec 5, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix bug
  • Loading branch information
@morningman
morningman committed Dec 5, 2019
commit b29a47fa6b3661e64f776f25df4be2ec3f3bb262
4 changes: 3 additions & 1 deletion fe/src/main/java/org/apache/doris/load/loadv2/LoadJob.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -628,7 +628,9 @@ private void executeFinish() {
Catalog.getCurrentGlobalTransactionMgr().getCallbackFactory().removeCallback(id);
state = JobState.FINISHED;

MetricRepo.COUNTER_LOAD_FINISHED.increase(1L);
if (MetricRepo.isInit.get()) {
MetricRepo.COUNTER_LOAD_FINISHED.increase(1L);
}
}

protected boolean checkDataQuality() {
Expand Down
5 changes: 3 additions & 2 deletions fe/src/main/java/org/apache/doris/load/loadv2/LoadManager.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@
import org.apache.doris.common.io.Writable;
import org.apache.doris.common.util.LogBuilder;
import org.apache.doris.common.util.LogKey;
import org.apache.doris.common.util.TimeUtils;
import org.apache.doris.load.EtlJobType;
import org.apache.doris.load.FailMsg;
import org.apache.doris.load.FailMsg.CancelType;
Expand Down Expand Up @@ -677,8 +678,8 @@ public void fixLoadJobMetaBugs(GlobalTransactionMgr txnMgr) {

// use CancelType.UNKNOWN, so that we can set finish time to be the same as the create time
job.cancelJobWithoutCheck(new FailMsg(CancelType.TXN_UNKNOWN, "transaction status is unknown"), false, false);
LOG.info("finish load job {} from {} to CANCELLED, because transaction status is unknown. label: {}, db: {}",
job.getId(), prevState, job.getLabel(), job.getDbId());
LOG.info("finish load job {} from {} to CANCELLED, because transaction status is unknown. label: {}, db: {}, create: {}",
job.getId(), prevState, job.getLabel(), job.getDbId(), TimeUtils.longToTimeString(job.getCreateTimestamp()));
}
}
}
Expand Down